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SECURITY 


SUMMARY 


Problem 


The  global  problem  addressed  by  this  research  Is  the  development  of  measurement 
technology  that  can  relate  enlistment  standards  to  actual  job  performance.  Within 
overall  effort,  it  is  necessary  to  develop  performance  measures  for  the  Navy  radioman 
(RM)  rating. 

Objective 

The  research  detailed  in  this  report  identified  a  set  of  critical  tasks  within  the  job  ox 
a  first-term  Navy  RM,  as  a  preliminary  requirement  for  developing  performance 
measures. 

Approach 

Task  identification  and  selection  was  accomplished  in  a  two-phase  approach  that 
included  job  analysis,  interview,  questionnaire,  card  sort,  data  analysis,  and  the  experi¬ 
ence-based  judgment  of  subject  matter  experts  (5ME)  approaches.  The  research  drew 
upon  the  judgments  of  both  RM  job  incumbents  and  supervisory  personnel.  Reliability 
analyses  and  factor  analyses  were  completed  on  questionnaire  response  data  and  on 
judgment-based  task  categorization  data.  Candidate  tasks  wer-r  recommended  to  a 
Quality  Control  Review  Panel  {QCRP},  a  group  of  specially  selected  Navy  job  experts 
cooperating  in  the  research,  which  made  the  final  task  selection.  Critical  task 
identification  and  selection  was  guided  by  Guion’s  (1979)  paradigm  for  reducing  the  job  to 
a  job  sample. 

Results 


The  research  necessary  to  reduce  the  job  to  a  job  sample  was  accomplished.  Data 
analyses  revealed  excellent  reliabilities.  The  selected  critical  tasks  form  the  basis  for 
test  items  that  will  be  developed  for  administration  in  the  operational  environment. 

Conclusions 


1.  Critical  tasks  for  first-term  RMs  were  identified  through  a  systematic  procedure  that 
included  description  of  the  job  content  universe  and  sampling  from  that  universe.  While 
there  is  some  possibility  that  one  or  more  tasks  might  prove  infeasible  for  hands-on 
administration— and  therefore  require  replacement— the  final  list  of  critical  tasks  repre¬ 
sents  a  sound  basis  for  item  development, 

2.  This  research  attempted  to  operationalize  assumptions  about  what  should  be  Included 
in  the  performance  test  and  to  use  them  in  the  job  analysis  and  task  selection.  To  a  large 
extent,  this  effort  was  successful:  The  task  selection  was  based  on  criteria  meaningful 
for  the  purposes  of  measurements. 

3.  The  procedures  used  to  obtain  the  final  job  sample  ensure  that  the  tests  will  have 
high  content  validity.  The  QCRP  reviewed  work  at  each  step  In  the  process,  assuring  a 
high  quality  product.  The  QCRP  procedure  also  established  an  audit  trail,  facilitating 
acceptance  of  the  final  test  package  by  the  commands  whose  representatives  are  on  the 
panel.  Data  indicate  that  reliable  judgments  were  obtained. 


v 


Recommendations 

U  Accept  the  critical  tasks  identified  in  this  research  as  a  valid  subset,  or  domain,  of 
the  test  content  universe  for  RMs. 


2.  Using  the  set  of  critical  tasks  identified  here,  proceed  to  develop  the  hands-on  job 
sample  test,  the  job  knowledge  simulation  test,  and  the  rating  scales. 

5.  Consider  applying  the  procedures  for  domain  definition  employed  in  tills  research  to 
the  development  of  performance  measures  for  other  Navy  ratings. 


JO#  CONTENT 
UNIVERSE 


JOB  CONTENT 
DOMAIN 


TEST  CONTENT 
UNIVERSE 


TEST  CONTENT 
DOMAIN 


Figure  2  (see  p.  $).  Task  identification  and  selection  for  testing  the  Navy  radioman.  The 
Joint  Service  Job  Performance  Measurement/Eniistment  Standards  Project  will  eventually 
link  enlistment  standards  directly  to  job  performance.  Current  standards  are  based  on 
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INTRODUCTION 


Problem 


In  response  to  Congressional  and  Department  of  Defense  (DoD)  concerns,  the  services 
are  investigating  measurement  approaches  that  might  make  personnel  classification  more 
performance  based.  A  previous  report  (Laabs  &  Berry,  in  press)  outlined  the  research 
strategy  of  the  Joint-Service  Job  Performance  Measurement/Enlistment  Standards  Pro¬ 
ject.  Basically,  that  strategy  is  to  construct  hands-on  performance  measures  and 
investigate  their  use  for  job  performance  predictor  validation.  A  major  research  focus  is 
the  development  of  job  sample  tests  that  can  be  used  as  a  high-fidelity  benchmark  against 
which  surrogate,  less  expensive,  measures  wili  be  compared. 

The  global  problem  is  the  development  of  measurement  technologies  accurate  enough 
to  relate  enlistment  standards  to  actual  job  performance.  Within  that  overall  effort,  it  is 
necessary  to  develop  performance  measures  for  the  Navy  radioman  (RM)  rating.  Because 
(I)  a  major  career  point  is  rvrched  by  survival  tc  the  end  of  the  first  enlistment,  and  (2) 
the  emphasis  is  on  apprentice-kvel  job  performance  prediction,  the  effort  reported  here 
focused  on  first-termers  (i.e.,  job  incumbents  with  48  months  or  less  service). 

The  RM  rating  was  selected  by  the  Navy  for  performance  test  development  because 
(1)  it  is  critical  to  mission  success;  (2)  it  has  a  large  population,  including  substantial 
numbers  of  women  and  ethnic  minorities;  ano  (3)  it  is  similar  to  RM  jobs  in  the  other 
armed  services  (Laabs,  Baker,  Kroeker,  &  Kidder,  1986).  Under  contract,  a  number  of 
performance  measures  are  being  developed  for  this  rating  including  (l)a  hands-on  job 
sample  test.  (2)  a  job  knowledge  simulation  test,  and  (3)  a  set  of  rating  scales. 

Objective 


The  objective  or  che  research  detailed  in  this  report  was  to  identify  a  set  of  critical 
tasks  subsumed  wit.Yn  the  job  of  a  first-term  Navy  RM,  as  a  preliminary  to  developing 
performance  measures, 

Background 

A  job  performance  measure  is  a  sample  of  stimuli  and  behavior  from  a  larger  domain 
of  job  content,  and  the  fidelity  with  which  this  sample  represents  the  job  content  domain 
is  a  serious  scient.fic  issue  (Brush  &  Schoenfeidt,  1982;  Distefano,  Preyer,  &  Erffmeyer, 
1983;  Guion,  1975),  Thus,  selecting  an  array  of  critical  tasks  is  the  fust  end  perhaps  most 
important  step  in  tost  development.  Unless  performance  measv*  . .  -  e  constructed  on  the 
basis  of  careful  cor  tent  sampling,  a  criterion-related  validity  study  becomes  an  infinite 
regress  argument  (Ehel,  !956,  1977;  Guinn,  1977).  Too  often,  the  definition  of  the  total 
job  content  and  task  sampling  are  haphazard  processes  in  which  the  objectives  for  the 
resulting  measure  are  never  made  explicit.  This  can  result  in  measures  of  low  utility. 

In  the  presem  case,  V,:  the  performance  measures  to  be  valid,  they  must  adequately 
represent  the  important  tasks  done  by  first- term  RMs.  In  fact,  the  validity  of  the  entire 
test  package  to  be  developed— the  hands-on  job  sample  test,  jcb  knowledge  simulation 
test,  and  the  rating  scales —depends  on  the  adequacy  of  the  methods  used  in  identifying 
and  selecting  the  behaviors  to  be  tested. 

Out  of  the  many  tasks  done  by  RMs,  a  manageable  subset  must  be  selected  as 
candidates  for  the  hands-on  test  items.  Test  items  will  be  critical  job  tasks  that  can 
feasibly  be  measured  in  the  hands-on  mode. 


A  Definition  of  ''Critical*' 


There  are  a  number  of  ways  in  which  one  might  define  "critical"  for  purposes  of  task 
selection.  One  might,  for  example,  select  the  tasks  that  have  the  most  dire  consequences 
in  the  case  of  inadequate  performance,  or  those  that  are  performed  most  frequently,  or 
those  on  which  the  largest  volume  of  training  resources  is  spent.  In  any  case,  the 
definition  should,  to  a  iarge  extent,  reflect  the  use  of  the  information  that  will  be 
gathered. 

The  current  application  of  the  data  involves  the  development  of  different  types  of 
performance  measures  to  ascertain  which  are  the  most  feasible  for  use  in  a  later,  more 
encompassing,  validation  effort.  One  of  the  types  of  measures  to  be  constructed  is  a  job 
sample  test,  which  involves  sufficient  expense  and  administrative  steps  to  limit  the 
number  of  tasks  that  may  be  tested.  For  that  reason,  a  target  of  15  tasks  was  established. 
This  rather  small  number  of  tasks  that  can  be  tested  makes  even  more  important  the 
earefui  selection  of  tasks,  taking  into  account  tl>e  users  of  the  tests,  the  amount  of 
information  expected  to  be  provided,  and  the  individual  duty  assignments  to  be  examined. 

Examinee  acceptance  was  essential.  That  is,  it  was  important  that  the  Navy  RMs  and 
the  test  scorers  view  the  tests  as  valid  measures  of  important  parts  of  the  job. 
Importance  in  this  context  includes?  (1)  performance  by  a  sizeable  number  of  RMs  and  (2) 
importance  to  mission  success. 

To  evaluate  performance  measures,  scientists  need  variance  in  task  performance. 
Consequently,  it  is  desirable  to  select  tasks  that  are  neither  so  hard  nor  so  easy  that  little 
information  is  gained  by  measuring  performance  on  them. 

Finally,  in  light  of  the  exigencies  of  Navy  work,  it  was  necessary  to  select  tasks  th3t 
cut  across  different  types  of  Navy  RM  assignments,  including  ship,  shore,  large  and  small 
operational  units,  and  so  on. 

The  definition  of  "critical”  that  emerged  may  be  summed  up  as  follows:  Critical 
tasks  are  those  that  are  (1)  performed  by  a  sizeable  number  of  incumbents,  (2)  important 
to  mission  success,  (3)  characterized  by  at  least  moderate  performance  variance,  and  (4) 
subsumed  in  a  iarge  number  of  Navy  RM  jobs  that  cut  across  varied  duty  assignments. 

The  Notion  of  Utility 

Determining  the  job  content  domain  necessarily  involves  tradeoffs.  One  such 
tradeoff  is  between  two  facets  of  representativeness  identified  by  Fitzpatrick  and 
Morrison  (1970:  fidelity  and  comprehensiveness.  Fidelity  is  the  realism  with  which 
individual  domain  elements  are  reproduced  in  the  test.  Comprehensiveness  is  the  number 
of  domain  elements  represented.  The  tradeoff  between  these  two  may  be  illustrated  by 
hands-on  measures,  which  produce  maximal  fidelity  but  are  so  expensive  to  develop  and 
administer  that  they  are  generally  limited  in  comprehensiveness. 

When  tradeoffs  are  involved,  decisions  must  reflect  utilities  based  on  values  assigned 
to  different  outcomes.  Utility  in  testing  has  been  largely  confined  to  (I)  predictors  and 
(2)  criterion-related  methods.  However,  l:  should  be  remembered  that  utility,  broadly 
defined,  is  the  payoff  the  organization  receives  from  a  measurement  effort,  whether  that 
effort  results  in  a  predictor  that  has  been  validated  by  a  criterion-related  method  or  a 
performance  measure  that  has  been  constructed  with  content-oriented  techniques.  A 
number  sf  factors  may  increase  or  decrease  the  utility  of  a  performance  test.  Basing  it 


on  tasks  that  are  critical  to  organizational  success  tends  to  increase  the  utility,  as  would 
using  the  resources  to  measure  performance  on  areas  of  the  job  that  are  not  performed 
well  by  ail  incumbents.  A  successful-hurdles  approach  (i.e.,  an  approach  based  on  a 
particular  sequence  of  events)  to  this  problem  can  help  ensure  that  the  various  utility 
considerations  have  received  appropriate  weight  in  the  task  selection  process. 


GENERAL  APPROACH 

Critics!  task  identification  and  selection  was  accomplished  by  an  approach  that 
included  job  analysis,  interview,  questionnaire,  card  sort,  data  analysis,  and  the  experi¬ 
ence-abased  judgment  of  subject  matter  experts  (SMEs).  The  research  drew  upon  the 
judgments  of  both  RMs  and  their  supervisors.  Candidate  tasks  were  recommended  to  a 
Quality  Control  Review  Panel  (QCRP),  a  group  of  specially  selected  Navy  job  experts 
cooperating  in  the  research,  which  made  the  final  task  selection. 

Critical  task  identification  and  selection  was  guided  by  Guion's  (1979)  paradigm  for 
reducing  the  job  to  a  job  sample.  It  includes  four  entities  often  confused  or  not  clearly 
defined  in  discussions  of  content  sampling  methodology.  A  basic  distinction  in  the  model 
is  made  between  a  universe  and  a  domain.  A  domain  is  generally  a  nonrandom  sample 
from  a  universe  (Figure  l).  Guion's  four  major  steps  include  determining  the  (i)  job 
content  universe,  (2)  job  content  domain,  (3)  test  content  universe,  and  (9)  test  content 
domain.  Our  research  addressed  these  four  steps  in  two  phases,  the  first  dealing  with 
selecting  job  content  and  the  second  with  selecting  test  content,  as  shown  below. 


Job  content  universe 
Job  content  domain 

- Test  content  universe 

— - Test  content  domain 


Figure  1.  Venn  diagrams  relating  job  content  to  test  content, 
labelled  for  Phase  of  Effort  (adapted  from  Guion,  1979). 
Test  content  includes  nonjob  elements  such  as  scoring 
procedures  and  rules  for  generating  test  stimuli  and 
responses. 


The  critical  tasks  selected  form  the  basis  upon  which  the  test  items  will  be  developed 
for  administration  in  the  operational  environment.  Figure  2  shows  the  steps  we  used  to 
adapt  Guion's  (1979)  schema.  I?  shows  the  orderly  reduction  of  job  task  information 
contained  in  the  job  content  universe  to  the  final  set  of  15  tasks  selected  for  test 
development,  indicating  the  major  sources  of  information  used  in  each  step. 
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TEST  CONTENT 
UNIVERSE 


TEST  CONTENT 
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Figure  2.  Task  identification  and  selection. 


PHASE  ONE:  SELECTING  3QB  CONTENT 


Step  1:  Defining  the  3ob  Content  Universe 

The  first  component  of  the  Guion  model  is  the  job  content  universe,  which  consists  of 
all  the  tasks,  duties,  and  characteristics  of  the  job.  The  constrained  job  content  universe 
with  which  this  effort  was  concerned  is  that  of  the  first-term  RM  job,  whether  located 
ashore  or  at  sea,  at  a  large  or  small  installation. 

To  develop  the  task  list,  the  research  staff  began  by  reviewing  job  documentation 
such  as  previous  job  analyses,  training  materials,  and  so  forth.  The  RM  job  content 
universe  was  defined  by  a  comprehensive  task  analysis.  Two  major  sources  of  information 
were  used:  data  from  the  Navy  Occupational  Task  Analysis  Program  (NOTAP),  which  is 
used  to  develop  the  RM  occupational  standards,  and  the  3ob  Task  Inventory  (3TI),  which  is 
used  in  developing  the  entry-level  training  (i.e.,  "A*'  school)  RM  curriculum. 

The  two  sources  overlap  considerably  and  contain  500-600  different  task  statements. 
Examples  of  tasks  from  each  source  are  listed  in  Figure  3. 

To  supplement  the  information  from  these  two  sources,  we  interviewed  first-term 
RMs  and  their  supervisors,  in  which  the  major  task  areas  of  the  job  were  defined  and  the 
component  tasks  of  each  task  area  were  detailed.  We  augmented  the  interviews  with  job 
observation. 

We  wrote  descriptions  at  a  level  of  specificity  that  would  facilitate  feasible, 
meaningful  units  for  hands-on  testing.  To  ensure  that  all  task  statements  were  written  at 
the  same  level  of  generality,  subtasks  were  accumulated  into  more  comprehensive  tasks, 
while  broader,  more  general  tasks  were  separated  into  their  constituent  parts.  The 
original  list  included  weli  over  500  tasks  that  defined  the  job  content  universe. 

Step  2:  Determining  the  3ob  Content  Domain 

Next,  the  job  content  domain  for  first-term  RMs  was  specified.  The  job  content 
domain  is  a  sample  from  the  job  content  universe  that  has  been  chosen  for  testing 
according  to  some  predetermined  objective(s).  In  this  case,  only  tasks  in  the  technical 
proficiency  domain  were  selected. 

The  joint-service  research  strategy  addresses  technical  job  proficiency  only,  speci¬ 
fically  excluding  purely  military  or  routine  administrative  tasks,  as  well  as  other  job 
performance  considerations  such  as  team  tasks  and  motivational,  situational,  cr  stress 
factors.  Since  the  purpose  of  the  project  is  to  construct  measures  tapping  first-term  RM 
specialty-specific  performance,  only  those  tasks  performed  uniquely  by  first-termers  were 
of  interest:  Navy-wide  tasks  such  as  cleanup  and  damage  control  were  not.  The 
refinement  process  deleted  Navy-wide  tasks,  as  well  as  those  not  within  the  normal 
purview  of  first-termers. 

Initial  Task  Selection 


Method.  The  original  list  of  task  statements  was  refined  and  reduced  through  two 
workshops  that  included  SMEs  who  were  supervisors  of  first-term  RMs.  Both  ship  and 
shore  installations  were  represented.  Seven  senior  Navy  enlisted  personnel  and  one 
civilian  attended  the  first  workshop,  which  was  held  in  San  Diego.  A  second  workshop, 
held  at  Norfolk,  Virginia,  in  which  seven  senior  Navy  enlisted  personnel  participated, 
further  shortened  the  task  list. 
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Examples  of  tasks  from  NOTAP  and  TTI  sources. 


Results.  Through  this  refinement  process,  124  task  statements  were  initially 
selected,  these  were  evaluated  by  the  QCRP,  which  approved  a  final  list  of  127  task 
statements  (the  panel  added  3  tasks  to  the  original  these  tasks  specifically  address 
technical  proficiency. 

Identification  of  ?ob  Categories 

Method.  To  help  later  selection  of  \ iest  content,  an  additional,  parallel  step  of 
categorizing  the  tasks  along  underlying  job  performance  dimensions  was  required. 
Sampling  from  what  wiii  later  become  the  test  content  domain  along  underlying 
dimensions  ensures  adequate  job  content  representation.  During  this  step,  job  categories 
were  identified  in  a  workshop  setting. 

In  order  to  derive  the  job  categories,  a  taxonomy  of  tasks  was  developed  at  a 
workshop  attended  by  12  senior  staff  personnel  from  the  Navy's  entry -level  RM  training 
course  in  San  Diego. 

Card  decks  were  used  to  identify  the  job  categories  for  stratified  sampling  of  critical 
tasks.  Each  task  was  placed  on  one  card.  (At  the  time  that  this  sorting  task  was  dene, 
the  task  list  consisted  of  132  statements,  S  of  which  were  later  deleted.)  Thirteen  R.M  job 
experts  then  sorted  the  cards  into  categories  of  the  same  type  or  that  dealt  with  the  same 
type  of  equipment.  The  instructions  for  this  task,  adapted  from  Borman,  Toquam,  and 
Rosse  (1977),  did  not  restrict  the  number  of  categories  that  could  be  used.  The  sort 
yielded  the  following  categories:  (1)  preparing  and  processing  messages/estabiishing 
communications,  (2)  setting  up  equipment,  (3)  maintaining  equipment,  and  (4)  handling 
secure  materials. 

To  achieve  a  metric  multidimensional  scaling  solution,  the  individual  sortings  were 
first  cumulated  into  a  132  x  132  raw  similarity  matrix,  with  each  cell  ^  representing  the 

percentage  of  times  that  tasks  2  and  j[  were  placed  in  the  same  category  by  the  13 
respondents.  A  mean  interproduct  matrix  was  then  computed  from  these  similarities 
using  the  following  formula: 

132 

1  siksjk 

M.  =  r=i 


This  measure’s  advantage  over  the  raw  similarity  measure  is  that  it  takes  into  account  the 
patterns  of  similarity  across  all  tasks.  The  factor  analysis  of  similarity  indices  is 
described  in  Nunnaily  (1978)  and  Torgerson  (1965). 

This  matrix  of  mean  interproducts  was  then  factored,  and  the  factors  were  rotated 
using  the  varimax  criterion.  Tasks  were  assigned  to  factors  using  the  highest  loading,  as 
the  goal  was  to  sampie  tasks  from  content  areas.  The  number  of  factors  to  accept  was 
based  on  an  examination  of  the  magnitudes  of  values  in  the  factor  structure  matrix  for 
each  factor.  The  four  factors  accounted  for  97  percent  of  the  variance. 

In  forming  the  categories,  we  emphasized  interpretation  over  strict  adherence  to  the 
algorithm.  The  goal  was  to  identify  job  categories  only  as  broad  guidelines  for  task 
selection;  the  ultimate  task  selections  were  to  be  made  by  job  experts  using,  in  part,  the 
job  analysis  questionnaire  data.  The  data  analysis  was  thus  used  only  as  a  tool  to  aid 
experts'  interpretation  of  the  job. 


Rcsui'is,  An  orthogonal  four-factor  solution  was  chosen  based  cn  interpretability  and 
n'lagmtuaeTof  factor  loadings,  and  the  factors  were  interpreted  by  the  items  assigned  to 
each*  The  names  of  the  four  factors  and  the  numbers  of  tasks  assigned  to  each  are  as 
foiicwsi  preparing  and  processing  messages/establishing  communications  (69),  setting  up 
equipment  (2 %  maintaining  equipment  (21),  and  handling  secure  materials  (11).  As  an 
attempt  to  increase  the  interpretability  of  the  results,  other  procedures  for  assigning 
tasks  to  factors  were  attempted  (such  as  using  all  the  factor  loadings);  however,  this  did 
not  increase  interpretability.  Eight  tasks  were  subsequently  eliminated  based  on  the 
QCRP  recommendations,  leaving  124. 


PHASE  2t  SELECTING  TEST  CONTENT 

Step  3?  Defining  the  Test  Content  Universe 

The  next  step  in  the  procedure  is  to  identify  the  test  content  universe,  which  in 
addition  to  tasks,  consists  of  rules  for  generating  test  stimuli  and  responses,  scoring 
procedures,  and  conditions  for  measurement.  Put  simply,  this  universe  embraces  all  tasks 
that  could  be  included  in  a  hands-cn  test,  plus  elements  introduced  by  the  testing 
situation. 


The  testing  situation  includes  conditions  imposed  to  achieve  relatively  standardized 
testing  and  the  procedures  used  to  observe  and  record  responses.  For  the  RM  rating,  the 
test  content  universe  Is  essentially  the  same  as  the  job  content  domain,  defined  by  the  127 
tasks  identified  in  Phase  1,  except  for  the  addition  of  the  testing  conditions.  These 
testing  conditions  will  have  to  be  specified  after  test  item  development  and  refined  after 
the  pilot  test. 

A  way  must  be  found  to  reduce  the  test  content  universe  of  127  tasks  to  manageable 
proportions.  Some  priorities  must  be  established  to  determine  test  content  domain  (i.e., 
task  selection  for  test  development).  One  of  the  typical  methods  to  set  task  selection 
priorities  is  to  gather  information  or>  the  criticality  of  performing  each  task  correctly  and 
the  frequency  with  which  it  is  done.  During  this  step  a  questionnaire  was  developed  and 
administered  to  gather  the  information  required  to  determine  the  test  content  domain  in 
Step  4. 

Method 


Questionnaire  development.  The  Navy  conducted  an  extensive  survey  to  gather 
judgments  from  personn^hTthe  rating.  The  327  task  statements  that  define  both  the  job 
content  domain  and  the  test  content  universe  were  incorporated  in  a  questionnaire,  the 
Radioman  (RM)  Survey  Form,  designed  to  isolate  the  critical  first-term  RM  tasks. 

The  questionnaire  was  developed  in  two  forms,  both  based  on  the  same  task  list.  One 
was  for  first-term  RMs  (those  R Ms  with  4  or  less  years  of  active  duty),  and  one  for 
supervisors  (RMs  with  between  4  and  10  years  of  active  duty). 

The  scales  and  anchors  were  developed  in  discussions  with  job  experts.  The  QCRP 
had  proposed  four  judgment  scales:  (1)  frequency  of  task  performance  by  first-termers, 
(2)  difficulty  (complexity)  of  the  task,  (3)  importance  of  the  task  to  mission  accomplish¬ 
ment,  and  (4)  frequency  of  errors  on  t hi  task  by  first-termers. 

The  questionnaire  for  first-term  RMs  (Appendix  A)  asked  them,  first,  to  check  the 
tasks  they  perform  and  then,  for  each  task  checked,  to  rate  (I)  the  frequency  with  which 
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it  is  performed  and  (2)  the  complexity  of  the  task*  All  responses  were  made  using  the 
5-point  scales  shown  in  Appendix  A,  The  other  questionnaire,  for  supervisors  of  first- 
terrn  RMs  (Appendix  B),  asked  supervisors,  first,  to  check  those  tasks  performed  by  first- 
termers,  and  then,  for  each  task  checked,  to  rate  (i)  the  importance  of  the  task  for 
mission  success,  and  (2)  the  performance  errors  made,  or  the  percentage  of  time  tliat  the 
task  is  performed  incorrectly  when  it  is  done.  The  first  rating  scale  for  both 
questionnaires  was  designed  to  measure  task  importance,  the  second  to  measure  task 
variance. 

The  questionnaires  were  designed  for  self-administration,  with  completely  self- 
contained  instructions.  Random-response  items  were  included  to  identify  individuals  not 
taking  the  questionnaire  seriously. 

The  instructions  and  rating  scales  were  refined  based  on  two  pretests  with  20  RMs 
and  supervisors.  One  major  change  was  that  the  scales  were  changed  to  reflect  absolute 
rather  than  relative  judgments,  with  the  review  panel  providing  the  scale  values,  or 
anchors.  That  is,  judgment  was  made  against  an  external  scale  with  each  level  defined, 
rather  than  vis-a-vis  other  items,  judgments,  etc. 

Questionnaire  administration.  The  QCRP  assisted  in  developing  the  sampling  plan  for 
distribution  of  the  Survey  forms.  Panel  consensus  was  that  the  jobs  of  first-term  RMs 
probably  differ,  depending  primarily  on  whether  the  job  is  in  a  large  or  a  small 
communications  facility.  Other  differences  might  occur  depending  on  the  job  site:  that 
is,  ship  or  shore  installation.  Therefore,  hull  types  and  shore  installation  types  were  both 
dichotomized  by  the  panel  into  large  and  small  categories.  The  review  panel  asserted  that 
there  is  no  difference  whatever  in  the  RM’s  job  between  overseas  and  continental  U.S. 
locations. 

The  questionnaire  administration  was  designed  using  a  proportional  sampling  plan 
based  on  the  number  of  RMs  and  supervisors  in  each  of  four  samp?'ng  cells  defined  by 
these  distinctions:  ship  versus  shore,  and  large  versus  small  insta.lation  (for  ship  and 
shore).  The  large  versus  small  installation  comparison  was  of  interest  within  both  ship  and 
shore  groups. 

Using  the  Enlisted  Master  Tape,  the  total  number  of  RMs  (both  first-term  and 
supervisor)  assigned  to  each  Navy  facility  categorized  by  the  review  panel  was  deter¬ 
mined.  Population  totals  in  each  of  the  four  cells  (made  by  crossing  facility  size  and 
sea/shore  location)  were  used  to  determine  the  number  of  questionnaires  to  be  mailed. 
(Submarines  were  excluded  from  the  selection  process  to  minimize  adverse  impact  on 
operations.) 

The  proportion  of  personnel  who  performed  the  tasks  was  the  most  widely  varying 
aspect  revealed  in  the  data  collection.  Because  of  this,  the  simple  random  sampling 
formula  for  binomial  data  (with  correction  for  finite  populations)  was  applied  to  each  cell. 
This  ensured  an  adequate  sample  for  each  of  the  four  potential  job  subtypes,  should 
separate  tests  later  prove  necessary;  that  is,  if  a  common  core  test  cannot  be  achieved. 

Because  each  cell  includes  a  number  of  different  types  of  ship  or  shore  installations, 
the  sample  was  drawn  proportionally  within  each  cell  according  to  installation  type.  The 
PQ  split  (where  P  =  probability  of  endorsement  of  a  response,  and  Q  =  probability  of 
nonendorsement  of  that  response)  used  in  the  sampling  formula  was  worst  case  (i.c.,  .5), 
the  level  of  tolerance  was  .10,  and  the  probability  level  set  at  .95.  Thus,  the  number  of 
personnel  surveyed  in  each  cell  was  determined  by  the  formula: 


N  *  T7n7P^3atl5r 

The  final  questionnaire  was  sent  to  approximately  500  first-term  RMs  and  500 
supervisors  throughout  the  world.  To  facilitate  timely  return  of  the  questionnaires, 
personnel  assigned  to  ships  that  were  deployed,  on  overseas  cruises,  or  in  overhaul  status 
were  eliminated.  Except  for  this  restriction  and  the  exclusion  of  submarines,  sampling 
was  random. 

The  questionnaires  were  addressed  to  individuals  by  name  and  delivered  via  their 
commanding  officer,  along  with  a  letter  summarizing  the  purpose  of  the  project  and 
explaining  that  their  response  was  important.  Receiving  commands  were  provided  with  an 
envelope  for  returning  the  questionnaires  to  research  staff. 

Results 


Table  1  presents  the  number  of  job  analysis  questionnaires  sent,  returned,  and  deemed 
useable  by  the  screening  for  response  integrity  (whether  the  questions  were  taken 
seriously).  The  mean  return  rate  across  sampling  cells  was  77.50  (range  73-84),  but  the 
mean  useable  return  rate  after  screening  was  52.75  (range  46-62).  While  no  obvious 
differences  could  be  discerned  between  those  who  did  return  useable  questionnaires  and 
those  who  did  not,  the  rather  low  number  of  useable  questionnaires  should  be  noted. 

Table  1 

Sampling  Information  and  Return  Rates  for  3ob  Analysis  Questionnaires 


Group 

Sent 

Returned 

Useable 

Percent  Returned 
and  Useable 

First-term  RM 

Shore 

Large 

140 

117 

69 

49 

Small 

120 

91 

55 

46 

Ship 

Large 

137 

109 

77 

56 

Small 

142 

105 

88 

62 

Supervisors 

Shore 

Large 

138 

112 

72 

52 

Small 

119 

92 

59 

50 

Ship 

Large 

114 

85 

52 

46 

Small 

132 

97 

81 

61 
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The  first  step  in  data  analysis  was  the  response  integrity  check  on  all  questionnaires 
to  screen  out  those  with  questionable  response  patterns.  Intraclass  correlations  were 
calculated  within  sampling  cells  to  assess  the  reliabilities  of  the  questionnaire  responses. 
The  following  were  computed  for  each  task:  percentage  performing  (incumbents)  or 
supervising  (supervisors),  both  overall  and  within  sampling  cells;  and  mean  values  on  all 
response  scales,  both  overall  and  within  sampling  cells.  To  assess  variability  between 
sampling  cells,  point  biserial  correlations  were  computed  for  differences  between  ship  arid 
shore,  large  ship  and  small  ship,  and  large  shore  and  small  shore. 

The  percentages  performing  or  supervising  and  the  mean  values  on  the  response 
scales  from  the  job  analysis  questionnaire,  both  overall  and  within  sampling  cells,  were  to 
be  used  in  critical  task  selection.  Since  critical  task  selection  is  made  relative  to  a 
number  of  possible  criteria,  specific  rules  were  not  established  prior  to  examining  the 
data.  However,  the  following  served  as  guidelines:  (1)  Tasks  should  be  performed  by  a 
large  percentage  of  the  job  incumbents,  (2)  critical  tasks  should  be  among  the  most 
important  and  frequent  as  viewed  by  questionnaire  respondents,  and  (3)  tasks  should  be 
intermediate  in  difficulty  such  that  they  are  neither  too  difficult  nor  too  easy  for  the 
typical  incumbent.  It  was  the  intent  to  apply  these  rules  for  both  the  overall  sample  and 
within  each  sampling  cell;  thus,  the  goal  was  to  select  tasks  meeting  the  above  criteria  in 
all  sampling  cells.  Selecting  in  this  way,  if  the  data  permit,  produces  a  set  of  criticai 
tasks  with  maximum  utility  to  the  Navy. 

Reliability.  Intraciass  correlations  were  computed  on  the  importance  and  frequency 
scale  responses  to  assess  reliability.  On  these  seales,  when  a  t33k  is  not  performed  or 
supervised,  a  "0"  rating  may  be  assumed;  and  there  is  thus  no  missing  data  (ratings  were 
made  wily  for  tasks  performed  or  supervised).  This  Is  not  the  case  for  performance  errors 
and  for  complexity:  A  "0"  cannot  be  assumed  if  the  task  is  not  checked  as  performed  or 
supervised.  As  a  result  of  this  difficulty,  reliability  analyses  were  not  conducted  on  these 
two  latter  scales. 

intraclass  correlations,  for  both  single  raters  and  composites  (corrected  with  the 
Spearman-Brown  formula),  are  shown  in  Table  2.  The  means  for  each  cell  on  each  task 
were  used  as  the  decision-making  unit  for  critical  task  selection.  Ail  reliability  estimates 
were  quite  high,  indicating  that  the  composites  on  which  criticai  tasks  were  selected  are 
quite  stable. 


Table  2 

Mean  Reliability  Estimates  for  3cb  Analysis  Questionnaires 


First-term  RM  Supervisors 


Ship  Size 

Shore 

Ship 

Shore 

Ship 

Large 

.96 

.94 

.97 

.96 

Small 

.97 

.94 

.97 

.95 

Note:  Cell  entries  are  intraciass  correlations  corrected  with 

Spearman-Brown  formula. 
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Step  4:  Determining  the  Test  Content  Domain 

The  test  content  domain  consists  of  the  actual  selection  of  measures  from  the  test 
content  universe',  this  is  the  selection  of  the  job  sample.  The  method  for  determining  the 
domain  will  incorporate  decision  rules,  concurrence  of  the  QCRP,  and  the  logistical  tasks 
associated  with  field  data  collection  and  analyses.  The  results,  and  the  consequent 
achievement  of  this  fourth  and  final  step  in  Guion's  (1979)  paradigm,  will  be  accomplished 
through  the  development  of  hands-on  test  items  for  the  operational  environment.  The  job 
sample  has  been  selected,  and  test  development  is  underway. 

The  decision  rules  that  would  select  the  best  set  of  tasks  were  formulated  by 
analyzing  the  questionnaire  data  by  the  procedures  outlined  above  and  examining 
percentages  performing  or  supervising  and  the  rating  scale  means.  These  values  varied 
considerably  between  job  categories;  the  m^ans  and  standard  deviations  of  task  means 
from  each  category  are  summarized  in  Table  3  according  to  the  questionnaire  ratings. 
There  are  two  significant  implications  about  these  data:  First,  the  job  categories 
captured  meaningful  differences  in  performance  patterns  for  different  parts  of  the  job 
and  hence  have  some  utility  for  representative  sampling  of  critical  tasks.  Second, 
because  of  for  the  large/small  and  ship/shore  categories,  different  decision  criteria  for 
critical  task  selection  will  have  to  be  applied  in  each  category.  Though  the  same  basic 
steps  could  be  followed  for  each  category,  the  criteria  at  each  step  may  have  to  be 
different  to  select  the  target  number  of  tasks. 

Method 


The  decision  sequence  for  task  selection  was  based  on  the  following  assumptions:  (I) 
Moving  from  general  (sample-wide)  information  to  more  specific  (within  cells)  information 
would  produce  the  most  generaiizable  set  of  tasks;  (2)  supervisors'  judgments  are  more 
valuable  than  those  of  first-term  RMs,  so  percentage  supervising,  importance,  and 
performance -error  values  should  generally  be  given  weight  ever  percentage  performing, 
frequency,  and  complexity;  and  (3)  the  best  determinants,  in  order  of  criticality,  are 
importance  and  frequency,  percentages  performing  and  supervising,  and  performance 
errors  and  complexity. 

As  noted  above,  the  technical  proficiency  task  iist  was  drawn  from  several  sources. 
To  sample  critical  tasks  representatively  from  different  areas  of  the  job,  a  target  number 
of  tasks  to  be  selected  from  each  job  category  (large/ small  and  ship/shore)  was 
determined.  This  was  based  or;  the  numbers  o?  tasks  assigned  to  each  category  in  the 
factor  analysis.  These  targets  were  proportional  to  the  number  assigned;  for  example,  if 
50  percent  of  the  tasks  were  assigned  to  3ob  Category  I,  we  planned  to  select  5f  oercent 
of  the  tasks  from  that  category  if  the  data  would  permit  that  number.  These  targets  were 
to  be  considered  guidelines,  not  quotas,  as  the  data  might  suggest  fewer  tasks  for  some 
categories  and  more  for  others.  The  goal  was  a  set  of  critical  tasks  without  sacrificing 
the  criticality  of  individual  elements  for  the  representativeness  of  the  entire  set. 

The  strategy  employed  was  a  multiple,  or  successive,  hurdles  exercise.  Figure  4 
summarizes  the  procedure. 

A  concrete  example  of  these  rules  in  application  is  shown  in  Appendix  C,  which 
outlines  the  decision  steps  for  choosing  the  eight  tasks  from  3ob  Category  i.  The  primary 
rank  ordering  of  tasks  was  done  on  the  basis  of  overall  criticality  means,  which  reflect 
that  the  most  general  information  for  that  scale  is  based  on  supervisors'  judgments  and 
incorporate  the  best  determinants  of  criticality.  Decision  steps  then  proceeded  to  more 
specific  information,  information  from  incumbents,  and  the  other  indicators  of  task 
criticality. 
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Table  3 


Summary  of  Job  Analysis  Questionnaire  Data  by  Job  Category 


Job  Category 

Item 

1 

Preparing  and 
processing 
messages/est. 
communications 

2 

Setting  up 
equipment 

3 

Maintaining 

equipment 

4 

Handling 

secure 

materials 

Number  of  Job  Tasks 

69 

23 

21 

11 

Percentage  Performing 

Mean 

44 

25 

19.5 

40.2 

Standard  Deviation 

21, 1 

14.1 

18.9 

11.6 

Percentage  Supervising 

Mean 

4S 

34 

27.2 

50.9 

Standard  Deviation 

20.1 

1S.S 

19.2 

11.4 

Importance  (Supe -visors) 

Mean 

3.25 

3.43 

3.33 

4.43 

Standard  Deviation 

.49 

.49 

.35 

,33 

Frequency  (Incumbents) 

Mean 

3.39 

2.55 

2.19 

2.99 

Standard  Deviation 

,66 

.59 

.47 

.58 

Perf.  Errors  (Supervisors) 

Mean 

1.5  4 

1.74 

1.32 

1.12 

Standard  Deviation 

.24 

.22 

,14 

.04 

Complexity  (Incumbents) 

Mean 

1.73 

2.03 

1.87 

1.67 

Standard  Deviation 

.26 

.22 

.24 

,14 
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1.  defect  task*  with  the  most  critic*!  riling*  ov*r*H 

2.  Discard  tasks  that: 

a.  have  a  tow  performance  frequency 

b.  are  performed  by  a  tow  percentage  of  the  sampSng  populations 

c.  have  a  few  percentage  o*  performance  errors 
&  are  characterized  as  tow  cotrptexSy 

3.  Ensure  proportional  representation  by  category 


Figure  4.  Successive  hurdles  for  final  task  selection. 


Results 

■  Imp  i 


Using  basically  this  same  sequence  of  applying  job  descriptors  but  varying  the  cutoffs 
as  appropriate,  given  category  differences,  13  tasks  were  designated  as  primary  recom¬ 
mendations  for  selection  and  9  as  secondary  recommendations.  The  distribution  of  tasks 
across  categories  is  shown  in  Table  4.  !t  is  clear  from  that  table  that  few  of  the  tasks  in 
Categories  2  and  3  merited  consideration  as  critical. 

Table  4 

Preliminary  and  Final  Critical  Task  Selection  by  Job  Category 


Job  Category 

Primary 

Preliminary 

Secondary 

Final  Selected 
Tasks 

i.  Preparing  and  processing 
messages/establishing 
communications 

3 

7 

14 

2.  Setting  up  equipment 

3 

0 

4 

3.  Maintaining  equipment 

1 

0 

3 

4.  Handling  secure  materials 

1 

2 

3 

Ultimately,  22  tasks  were  selected  as  critical.  This  set  consisted  of  most  of  the 
original  22  primary  and  secondary  tasks,  with  the  exception  that  3  tasks  were  dropped  and 
3  not  in  the  initial  set  were  added.  The  added  tasks  reflected  operational  requirements 
that  are  to  be  more  heavily  emphasized  in  the  future  by  the  Navy.  The  distribution  of 
selected  tasks  across  job  selected  is  also  shown  in  Table  4.  The  selected  tasks  are  listed 
in  Appendix  D. 


Expert  Judgment 

The  final  responsibility  for  critical  task  selector  was  given  to  a  review  committee  of 
Navy  iob  experts.  The  primary  and  secondary  task  recommendations,  along  with 
supporting,  data,  were  supplied  to  the  commit  tee.  The  committee  made  its  selections 
based  not  only  upon  the  data,  but  upon  feasibility  of  testing,  operational  requirements, 
availability  of  equipment,  testing  time,  and  so  forth. 

To  assist  the  QCRP  in  this  process.  K-dimimry  ';.$k  selections  were  made  using  the 
job  analysis  data.  An  overall  target  of  15  critic-'  *$ks  was  set;  however,  the  final 
number  depended  upon  the  outcomes  of.  the  ts  sr.,z-.ysl$,  Previous  experience  with 
hands-on  testing  in  other  Navy  ratings  indicted  Ji at  15  was  a  feasible  and  sufficient 
number  of  tasks.  3ob  experts  received  both  primar  and  secondary  task  recommendations 
(the  secondary  group  was  somewhat  less  preferred  fc?  ed  on  the  data  alone).  Based  on  the 
assignment  of  tasks  to  job  categories,  the  following  targets  were  determined  for  appoint¬ 
ment  of  the  15  critical  tasks  across  the  four  eaiegcries;  From  Category  1,  eight  tasks 
would  be  selected,  three  tasks  would  be  selected  from  both  Categories  2  and  3,  and  one 
ta  k  would  be  selected  from  Category  4. 

The  decision  criteria  were  flexible  only  up  to  a  certain  point.  It  would  not  be  prudent 
to  select  tasks  to  fill  a  category  quota  if  an  insufficient  number  of  tasks  in  that  category 
failed  to  meet  reasonable  criteria  ex  criticality.  For  exc.cple,  if  only  one  task  in  a 
category  was  performed  by  a  sizeab*?  somber  of  first-term  RMs  and  was  further  the  wily 
one  rated  as  especially  critical,  frequ^-.t,  and  so  forth,  that  task  should  be  the  only  one 
selected  from  that  category. 

Note  that  the  15  tasks  are  included  in  these  shown  in  Table  The  ?>a!  set  of  tasks 
was  based  on  a  melding  of  data  and  expert  judgment.  The  QCRF  carefully  scrutinized  the 
list  of  candidate  tasks  and,  based  on  their  accumulated  experience  in  the  Fleet,  made 
recommendations  for  cr  against  each  test  item. 

The  final  task  set  that  emerged  was  based  on  consideration  of  such  interactive 
complexities  as  time  constraints,  impact  on  operational  conditions,  equipment  require¬ 
ments,  feasibility  for  hands-on  testing,  and  the  intratest  compatibility  and  sequencing  of 
items.  Tasks  were  chosen  based  on  coverage  for  the  category,  representativeness  of 
actual  first-term  RM  duties,  and  feasibility  for  testing.  Several  i^ska  were  also  put  in 
logical  sequence  by  QCRP.  An  important  consideration  will  be  the  question  of  exactly 
wha  i  equipment  is  to  be  used  for  on-site  data  collection.  The  actual  hands-on  test  wiii  be 
administered  using  the  Communications  Team  Trainer,  which  duplicates  a  "radio  shack" 
and  uses  actual  equipment. 

By  adding  a  number  of  tasks,  then  carefully  sequencing  and  timing  the  tasks,  the 
QCRP  arrived  at  a  set  of  items  that  are  testable  on  current  inventory  equipment,  should 
not  ?-ove  inordinately  long  in  administration,  and  provide  very  realistic  job  coverage  lor 
the  lirst-term  Ril.  The  final  U si  of  22  tasks  is  contained  in  Appendix  D. 


CONCLUSIONS 

1.  The  critical  tasks  for  first-term  RMs  were  identified  through  a  systematic  procedure 
that  included  thorough  description  of  the  job  content  universe  and  sampling  from  that 
diverse.  While  there  is  some  possibility  that  one  or  more  tasks  might  prove  infeasible  for 
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hands-on  administration— and  therefore  require  replacement— the  final  list  of  22  critical 
tasks  developed  in  this  effort  represents  a  sound  basis  for  item  development, 

2.  This  research  attempted  to  make  explicit  the  assumptions  about  what  should  be 
included  in  the  performance  measures  and  to  operationalize  them  in  the  job  analysis  and 
task  selection.  To  a  large  extent,  this  effort  was  successful,  as  the  task  selection  was 
based  on  operationally  meaningful  criteria  relevant  to  the  purposes  of  measurement. 

3.  The  procedures  for  obtaining  the  final  job  sample  ensure  that  the  tests  will  have  the 
high  content  validity.  The  QCRP's  expert  review  of  work  at  each  step  assures  a  high- 
quality  product.  In  addition,  the  QCRP  procedure  established  an  audit  trail,  facilitating 
acceptance  of  the  final  test  package  by  the  commands  whose  representatives  are  on  the 
panel.  Data  Indicate  that  reliable  judgments  were  obtained. 

4.  The  job  analysis  procedure  used  in  this  content  sampling  appears  to  be  generalizable 
enough  to  be  useful  with  a  variety  of  different  Navy  ratings. 


RECOMMENDATIONS 

1.  Accept  the  critical  tasks  identified  in  this  research  as  representing  a  valid  subset  of 
the  test  content  universe. 

2.  Using  the  set  of  critical  tasks  identified  here,  proceed  with  the  development  of  the 
hands-on  job  sample  test,  the  job  knowledge  simulation  test,  and  the  rating  scales. 

3.  Consider  applying  the  procedures  for  domain  definition  employed  in  this  research  to 
the  development  of  performance  measures  for  other  Navy  ratings. 
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API’ENDIX  A 

NAVY  RADIOMAN  TO 8  DESCRIPTION  QUESTIONNAIRE 
FOR  FIRST-TERM  RADIOMEN 


NAVY  RADIOMAN 

OB  DESCRIPTION  QUESTIONNAIRE 


(FORM  A:  To  be  filled  out  only  by  first-term  Radiomen) 


Developed  by: 

Human  Resources  Research  Organization 
27857  Berwick  Drive 
Carmel,  CA  93923 

and 

Personnel  Decisions  Research  Institute 
43  Main  Street  Southeast,  Suite  405 
Exposition  Hall 
Minneapolis,  MN  55414 


A-l 


INTRODUCTION 


The  Navy  is  conducting  a  project  to  find  the  best  ways  to  measure  the 
job  performance  of  first-term  Radiomen  in  the  Navy.  This  questionnaire 
is  an  early  step  In  that  project.  Its  purpose  is  to  learn  more  about 
the  first-term  Radioman  job  from  those  who  know  it  best--f1rst~term 
Radiomen  and  their  supervisors. 

Please  take  the  time  necessary  to  carefully  consider  each  item  and 
answer  it  to  the  best  of  your  knowledge.  We  appreciate  your  help. 


PRIVACY  ACT  NOTICE 


Under  the  authority  of  5  USC  301  judgments  concerning  the  job  perfor¬ 
mance  of  first- term  Radiomen  are  requested  for  a  research  project  on 
performance  assessment.  TH&JN  FORMATION  PROVIDED  WILL  NOT  BECOME  PART 
OF  YOUR  OFFICIAL  RECORD,  NOR  WILL  IT  8E  USED  TO  MAKE  DECISIONS  ABOUT 
YOU, WHICH  WILL  AFFECT  YOUR  CAREER  IN  ANY  WAY.  It  will  be  used  by  the 
Navy  Personnel  Research  and  Development  Center  for  statistical  purposes 
only.  Return  of  the  questionnaire  constitutes  acknowledgment  of  these 
Privacy  Act  provisions. 

This  questionnaire  has  been  assigned  Report  Control  Symbol  OPNAV  5312- 
5(0T) ,  which  expires  30  June  1985. 
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GENERAL  INFORMATION 


Please  print  your  answers  in  the  spaces  provided  or,  where  applicable, 
place  a  check  mark  In  the  sag.  blank  that  best  applies  to  you. 

2.  Name:  _ _ _____ _ _ _  . 

2,  Social  Security  Number:  ___ _ _ _ 

3.  Complete  Work  Mailing  Address: 


4.  What  is  your  current  pay  grade? 


E-2 

E-3 

E-4 

_  E-5 

_ Other  (please  specify:  _ _ ) 

5.  How  long  have  you  been  in  the  Navy? 

_  Less  than  $  months 

_  6  months  to  1  year 

I _  1  year  to  18  months  (1  1/2  years) 

_  18  months  to  2  years 

_  Z  years  to  30  months  (2  1/2  years) 

_  30  months  to  3  years 

_  3  years  to  42  months  (3  2/2  years) 

_  42  months  to  4  years 

~ _ _  4  years  or  sore 

6.  What  is  your  current  assignment? 

_ Ship 

~  Shore  Installation 


enter  the  ship  type  and  hull 

number  (e.g.,  CVNS8,  FH0S3):  _ _ 


Go  on  to  the  next  page 
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If  vour  curcen-t-is.5 fgimt  nt_ii_Qn-Shore ,  check  the  type  below: 


_  General  Duty 

_  NAVCAMS  (also  check  one  of  the  following:} 

FI eet  Center 

"  Tech  Control  Facility  (including  SATCOM) 

_ Message  Center/Data  Base/Computer  Deck 

____  NAVCdttSTA  (also  check  one  of  the  following:) 

FI  set  Ccntsr 

Tech  Control  Facility  (including  SATCOM) 
Message  Center/Data  Base/Computer  Deck 

_ _ _  Receiver  Site 

Transmitter  Site 
NTCC 

_ Other  (please  specify:__ _ 

7.  How  long  have  you  been  in  your  current  assignment? 

_  less  than  3  months 

_  3  months  to  6  months 

_  6  months  to  9  months 

_  9  months  tc  1  year 

_  More  than  1  year 


Go  on  to  the  next  page 
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INFORMATION  ABOUT  JOB  TASKS 


This  section  contains  a  list  of  first-term  Radioman  job  tasks.  The 
task  list  was  developed  by  examining  job  descriptions,  talking  with 
Radiomen  and  their  supervisors,  and  observing  first-term  Radiomen  on 
the  job.  This  list  covers  *11  first-term  Radioman  jobs  at  all  instal¬ 
lations,  so  it  say  contain  some  tasks  that  you  do  not  do  on  your  job. 

In  this  section,  you  will  answer  three  questions  about  these  tasks: 

--Is  each  task  part  of  your  current  job?  For  each  that  is... 

--How  frequently  is  it  performed? 

--How  complicated  is  it  te  perforo? 

As  you  fill  in  the  questionnaire,  consider  only  the  work  you  do  in  your 
current  .lob  assignment.  Do  not  consider  tasks  you  performed  in  pre¬ 
vious  assignments. 


PART  OF  CURRENT  JOB  Judgment 

Read  the  first  task  and  decide  whether  It  is  part  of  your  current  job. 
If  the  task  Is  a  part  of  your  current  job,  place  a  check  mark  next  to 
the  task  In  the  column  labeled  "FART  OF  CURRENT  JOB".  If  the  task  is 
npt  a  part  of  your  current  job,  go  on  to  the  next  task  without  placing 
anything  in  the  PART  CF  CURRENT  JOB  column. 

Go  through  the  entire  list  of  tasks  in  this  way.  DO  NOT  FILL  IN  ANY  OF 
THE  OTHER  COLUMNS  UNTIL  YOU  HAVE  CONE  THROUGH  THE  ENTIRE  TASK  LIST  AND 
INDICATED  WHICH  TASKS  ARE  A  PART  OF  YOUR  CURRENT  JOB. 

Example 

Following  is  an  example  of  how  one  first-term  Radioman  indicated  which 
of  three  sample  tasks  are  part  of  his/her  current  job. 


«M*t 

i  •  an  *•»  mmhm 

t  •  mmmm  mm  wwi 

« »  mm 

ttEUMKKMMI 

m 

(•mnu 

(•WWB 

*•»*  mavami 

1.  £fK94t/6tco<ft  C#!l  tt&u. 

y r 

2.  Rig  «*rg*ssy  wteswes. 

3.  Sic? 

m i 
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This  Radioman  went  through  all  the  tasks  and  Indicated  which  ones  are  a 
part  of  his/her  job.  Because  Tasks  A  and  C  are  part  of  the  job,  a 
check  mark  was  placed  by  each  of  those  tasks  in  the  PART  OF  CURRENT  008 
col  wan.  Task  8,  "Rig  emergency  antennas,”  Is  net  part  of  this  Radio- 
nan^  job  so  he/she  did  place  a  check  nark  In  the  PART  OF  CURRENT 
008  column  by  Task  B. 

This  Radioman  correctly  nade  the  PART  OF  CURRENT  00B  judgment  for  all 
the  tasks  on  the  list  before  going  on  to  make  any  other  ratings. 

FREQUENCY  and  COMPLICATED  TO  PERFORM  Ratings 

When  you  have  finished  Indicating  which  tasks  are  part  of  your  current 
job,  go  back  to  the  beginning  of  the  list  and  complete  the  FREQUENCY 
and  COMPLICATED  TO  PERFORM  ratings  only  for  those  tasks  that  you  pre¬ 
viously  Indicated  to  be  PART  OF  CURRENT  JOB. 

The  FREQUENCY  and  COMPLICATED  TO  PERFORM  ratings  are  to  be  nade  by 
filling  In  the  cm  appropriate  rating  for  each.  A  different  rating 
scale  Is  used  for  each  of  these  ratings. 

FREQUENCY  Rating 

For  the  FREQUENCY  rating,  Indicate  how  many  times  you  perform  the  task 
m,  MRfikJMLdiXi- grU&jJMUBflft-  Use  the  following  rating  scale  to  rate 
FREQUENCY : 

1  *  SELDOM  (Less  than  once  per  week) 

2  •  OCCASIONALLY  (About  once  per  week) 

3  -  SOMEWHAT  OFTEN  (2-6  times  per  week) 

4  «  OFTEN  (About  once  per  day) 

5  *  VERY  OFTEN  (Mere  than  once  per  day) 

For  each  task  that  you  checked  as  PART  OF  CURRENT  JOB,  enter  the  jh& 
rating  (1  t©  5)  that  fctii  describes  how  many  times  per  week  or  day  the 
task  Is  performed. 

COMPLICATED  TO  PERFORM  Rating 

For  the  COMPLICATED  TO  PERFORM  rating,  Indicate  hew  complicated  the 
task  Is  in  terms  of  the  knowledge  and  skill  reauired  to  perform  It 
correctly.  Use  the  rating  scale  that  begins  on  the  next  page  for  the 
COMPLICATED  TO  PERFORM  rating. 


1  -  VERY  SIMPLE 


2  -  SIMPLE 

3  -  SOMEWHAT  COMPLICATED 

4  -  COMPLICATED 

5  *  VERY  COMPLICATED 

For  c*ch  task  that  you  previously  checked  is  PART  OF  CURRENT  JOB,  enter 
the  fins  rating  (1  to  S)  that  best  describes  how  complicated  the  task  is 
In  terms  of  the  knowledge  and  skill  required  to  perform  it  correctly . 

Example 

After  checking  all  of  the  tasks  that  are  a  part  of  his/her  job,  this 
Radioman  made  the  FREQUENCY  and  COMPLICATED  TO  PERFORM  ratings  flfl]*  for 
those  tasks  that  had  been  checked. 
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2.  Rig  emergency  »rttmus. 

3.  Slat  sassages. 

/ 

v3 

L^_J 

Task  A:  This  Radioman  encodes/decodes  call  signs  occasionally:  about 
o-nce  oer  week.  Thus,  a  *2*  was  written  in  for  FREQUENCY.  In 
terms  of  how  complicated  the  task  is  to  perform  correctly, 
this  Radioman  believed  that  it  is  as^h&kCggalj&a&d,  so 
he/she  assigned  a  COMPLICATED  TO  PERFORM  rating  of  "3"  to  the 
task. 

Task  B:  This  task  was  not  checked  as  PART  OF  CURRENT  -30B.  Therefore, 
the  Radioman  did  not  fill  in  the  FREQUENCY  and  COMPLICATED  TO 
PERFORM  ratings. 
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Task  C:  "Slotting  messages"  U  perf <*rmd  somewhat  ofUnt  about  2.  to  6 
times  per  week,  so  this  Radieaan  assigned  a  FREQUENCY  rating 
of  "3H.  The  task  was  considered  to  be  simple  In  terms  of  the 
knowledge  and  skill  required  to  perform  it  correctly,  so  a  "2" 
was  written  in  for  COMPLICATED  TO  PERFORM. 

Go  on  to  the  next  page  and  begin  the  questionnaire. 
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msjsemaasm 
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t>  stress 
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s  *  tot  mo  {lte«  tt*  rntHMj)  | 
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2.  Set  up  IF  transmitter*. 


2.  Set  up  KF  transmitters. 


3.  Set  «p  THF  transmitter*. 


4.  Sat  up  IF  receivers. 


S.  Sat  up  KF  recti vers. 


4.  Set  up  VHF  receivers. 


7.  5*t  up  OKF  transceivers. 


I.  Set  up  ¥KF  transceivers. 


9.  Sat  up  satellite  transceivers. 


10.  Set  up  tli«  SSt-l  satellite  rtctivtr. 


II.  Set  up  antenna  eeuplers/multicouplers. 


12.  Set  up  crypt*  equipment. 


13.  Sat  up  teletypes. 


14.  CM»J*  peper/rihbont  on  telatypci  end 

printer*. 


IS.  Patch  communications  equipment  pieces 
together. 


1*.  laise/lewer  antennas. 


17.  Asaemble/disassemfcl*  antenna*. 


18.  Plat  ship  petitions  for  antenna  assignments. 


If.  Load  program  tapes  in  magnetic  tape  made 


20.  Loed  program  tapes  in  peper  tape  rtadtr. 


21.  Perform  Ut*  starts  on  crypto  equipment. 


22.  Check  frequencies  for  usability. 


23.  Use  chirp  sounder  to  determine  frequencies. 


0*  nmn.  Hv  «na  0*  jm 
tmrter*  «k1(  t  **« 


This  1t»  is  for  ieypeaeh  purposes  only.  6o 
Mt  writ*  ratings  far  till*  ant. 


Conduct  radio  cheeks  (seeere/nenseciire). 


Sendee!  quality  control  tests  on  tranmlt 
frequencies • 


Read  journal  (or  channel)  leg  teletypes  to 
determine  system  status. 


Maintain  cowaunlcatlons  status  board. 


foil  messages  from  teletype  or  printer. 


Screen  incoming  messages  for  relevance* 
precedence,  security  classification,  etc. 


Operate  reperforator. 


Maintain  listening  watch  on  radiotelephone/ 
voice  deceits  (e.g.,  Harbor  Ce— on,  KICCM) 
and  determine  trften  call  sign  it  received. 


Receive  radiotelephone/voice  circuit  ms* 
sages  and  transcribe  to  hard  copy. 


log  entries  in  the  Radiotelephone  Log. 


Recognize  and  properly  comply  with  special 
mssage  handling  procedures  ("personal  for", 
?1sited  distribution,  high  precadence, 
etc.). 


Monitor  channel  number  continuity  f or  ms* 
sage  traffic. 


draft  broadcast  screen  requests  as  approp¬ 
riate. 


log  incoming  mssagts. 

— mw X— — W  mi  ><■!■  ■  .1  i  un>  ■  r»«i  i  ■  «  lH 

Log  mssagts  In  General  Message  log. 


Reproduce  mssages  on  photocopier  and  verify 
copy  quality. 
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41.  fee  r*rxfirj  v»  4»t*r»1at  Olsirlwtl** 

*r  ro«tt1*f  of  IftCMffef  muftt. 


W.  Im  lUNirt  MJ«t  Mtf 

CSSIC)  NmmmI  to  ecrrect  tfe  loteraol  <!*• 
trftetlt*  of  MiUfit  9*  tfe  t 14m  tlsplay 
YoTKlMlt  (WT). 


49.  tonally  root*  KtMfH  U  lyyHiU  fet- 
tliwtleos  {•.§,,  sic:  wtuftt,  pest  ao  rto 
fear*.  •to.). 


44.  Verify  eotpaiap  roops  tofts  for  caop1*te~ 
eest.  tccorscy,  foraot,  to  rsteastof  si foo¬ 
ter*  , 


41.  Not)  H-llK  Fom  («U  pitten  aossoye 
raltiN  form}. 


44.  Uf  atsuftt. 


47.  tator  OCR  Ftot  late  Optical  Ouracttr 
fetor  (OCR). 


40.  fee  flfee  Bisplay  Temlaal  (TOT)  te  trefele- 
•feet  to  correct  OCX  Form  tfet  km  too 
reject*. 


49.  fesam  SCI  ferw  fro*  Optical  Character 
fetor  (OCR)  to  recto  fmatftf  $1*m1 
fetors  [n*t)  to  tims  of  troMoUstee. 


10.  Prioritise  eotpolof  aesseytt  eccerilof  to 
prectooe*  to  tlaa  of  receipt. 


it,  Verify  motk  mtUfts  trftk  sap enrioor/ 
reletting  officers  prior  t*  treacsUslee, 


42.  Proof rto  eatfetaf  fesstfes  prior  to 
trsosRissloa. 


U,  fecto  tie*  of  Velteery  to  eenrico  cross  oo 
wtujei. 


m  at  twwp,  tm  Oat  *t  ftc 
«*1  tMt! 

t  *  mm  (Uni  mm  mmtmrt) 
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W.  FfKtt*  wwtiei  tKWft  trefffe  iftrftrm 
met  cmmUt  Oetomis*  catty,  ecwpt  fwft, 
eu.)  «*4  stfeUia  eteeefate*  filet/rapart*. 


H.  Claee  Mt/uMMtt  *11  nrtf*  efreaft  it 

cfctaft  *f  rail*  $•/. 


St.  Pwp*re/tra««1t  ZIC/ZI*  mwrti. 


57.  Hefetafa  caaaMs!cetie**  «*Ur  anuf« 
ftlM. 


it.  Itef state  jeaerat  Miuft  filet. 


ft.  Meiatsie  iraarfcaet  fflex/Teft. 


59,  Jfefatafa  MtUfi  *#$*etlc  Up*  filet. 


it.  Select/!**  releveat  yeatrtl  ceeHanicetteas 
pahHeeiffcj,  fftstmctlMSe  m<  iirectiaat. 


$2.  MU  correct i*««t  te  m*  H9*  chick  cwaw.1- 
cstteet  pAltciffaxy. 


53.  hMtUrjf  yceeril  tom wxfcstietti  pabltct* 
t(wt. 


54.  Tjpe/feraet/etfft  aetMpes  on  teletype. 


IS.  Typ*/f*r*et/e4it  aesstfex  «?  Yt$*«  Oitpltjr 
Teraiwl  (m). 


tt.  Settraiae  ipprepHite  faratt  far  eutpaiae 
s*!**p:. 


.  Type  Mtuyit  AC7-1ZS  forait. 


IS.  Type  actsifO  vtfnf  AC*- 12$  fcllflH  foraet. 


IS,  Tips  muftt  estep  4Wf  12S  feraet. 


70.  Type  aettif«»  wis?  AC7-127  {RATOJ  forwt. 


71.  Type  mtupe*  esiftg  *ml  Comspomtoac* 
{litter,  0096)  ferwt. 
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71,  Type  *M  farmafc  messages  fcr  Optical  Cfeer- 
»et»r  Header  (OCX)  «si»j  tyywlUr. 

73.  Lead  ptper  tape  Mtttftt  late  Tide#  Bisplay 

Ttrarfiwl  ffST)  f er  editing. 

74.  Select  MM  ehaage  te  appntfrlflt*  mM  mi  TOT 
{fwrji  raster,  servlet,  etc.}. 

75.  Iaitfalfze  CUBIT,  links  vsinf  carnal  tele¬ 
type  er  terminal. 

78.  Cstabllsh/mafnUln  ccmmewicittows  ea  sh.p- 
Shore,  ship-ship.  «r  termination  circuits. 

77.  IftitlsU/respemd  te  challenge  eel  refly 
prece&re  ee  radiotelephone  circuits. 

7*.  Initiate  traasmistlcm  autheatieatiom  fra¬ 
ctures. 

71.  Establish/aalataln  communications  oa  task 
f reef  0*£STtS. 

10.  later  commands  e«  YOT  to  establish  YFCT  cir¬ 
cuit  configurations  ini  channel tzat Ions. 

SI.  Coordinate  ec—ur.icatiens  using  pony  loop* 
er  crier  wires. 

82.  Operate  air  preheater  controls. 

83.  Irani* It  messages  via  YOT  terminal. 

|  84.  Transmit  paper  Uje  messages  using  high, 
speed  tape  reader  and  comesaI  teletype 
(C03U  system). 

85.  Transmit  messages  via  radioteletype. 

85.  Transmit  messages  rU  radiotelepfcone/voice 
circuit. 

87.  gncode/decode  radio  frequencies,  short  sen¬ 
tences,  and  voice  call  signs  using  approp¬ 
riate  publications. 
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8i.  Receive  confidential  Mteriels  (excluding 
CHS). 

81.  Inventory  confidential  Mtertais  (excluding 
06). 

90.  Rest#  esnffdentlal  ntiHils  (excluding  06} 
to  appropriate  personnel. 

91.  Store  confidential  Mttrlalt  (excluding 
06}. 

92.  Destroy  confidential  Mtorials  (excluding 
06). 

93.  Receive  secret  Mteriels  (excluding  06). 

94.  Inventory  secret  Material*  (excluding  06), 

95.  Store  secret  Materials  {excluding  06). 

98.  Route  secret  Mteriels  (excluding  06/  to 
appropriate  personnel. 

97.  Destroy  secret  Mteriels  (excluding  06). 

ii  ■  ■  ■  ■  — — — .  ■  ,  —  -n.  . -I  _  . . 

9$.  Receive  top  secret  Materials  (excluding 
06). 

39.  Witness  destruction  of  top  secret  Materials 
(excluding  06). 

ISO.  Hitmn  the  receipt  ef  06  sstecisU. 

101.  Witness  the  inventory  of  06  mteriels, 

102.  Witness  the  control  of  06  Materials. 

103.  Witness  the  storing  of  06  Mteriels. 

104.  Store  06  Mteriels. 

105.  Witness  the  destruction  of  06  Materials. 

106.  Chech  Message  recipients'  identification  end 
security  clearance  to  ensure  proper  class!' 
ficsticn  for  level  of  traffic  being  picked  up. 


i07.  This  1  ten  1*  for  keypunch  purposes  only.  Do 
net  write  ratings  for  this  one. 


108.  Perform  preventive  maintenance  on  receivers 
(using  Maintenance  Record  Cards,  or  HFtCs). 


109.  Perform  preventive  maintenance  cn  trans¬ 
ceivers  (using  MRCs). 


110.  Perform  preventive  seintenance  on  transmit¬ 
ters  (using  MRCs). 


111.  Perform  preventive  maintenance  on  antennas 
(using  MRCs). 


112.  Perform  preventive  maintenance  on  antenna 
coup lers/muUi couplers  (using  MRCs). 


113.  Perform  preventive  maintenance  on  teletypes 
(using  MRCs). 


114.  Perform  preventive  maintenance  on  switch¬ 
boards  (using  HRCs). 


115.  Perform  preventive  maintenance  on  patch 
panels  (using  MRCs), 


'  116.  Perform  preventive  maintenance  on  reperfor- 
I  Itors  (using  MRCs). 


117.  Perform  preventive  maintenance  on  paper 
shredders  (using  MRCs), 

r^»  I  . I"**'  IU  ■■■  '  1 

118.  Perform  preventive  maintenance  on  copying 
machines  (using  MRCs). 

1,1 ,  m rmrr  ~i —  ■  mm ‘  —  "**  **“  11 1  •  1  HUMHW***—1 M»«rmw«i*i<iirui  ■!  w 

119.  Perform  preventive  maintenance  on  Optical 
Character  Readers  (or  OCRs,  using  MRCs). 


120.  Perform  preventive  maintenance  on  remote 
radio  telephone  operating  positions  (using 
MRCs). 


121.  Perform  preventive  maintenance  on  emergency/ 
portable  radio  equipment  (using  MRCs). 
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122.  Pe rfora  preventive  Maintenance  on  automated 
■esstge  reproductfon/distribution  systems 
(using  JSCs). 

123.  Clean  and  inspect  tape  heads. 

124.  Perfona  preventive  aicintenance  on  telegraph 
telephone  signal  converters  (using  HRCs). 

125.  Perform  preventive  Maintenance  or.  radio  sets 
(using  fttCs). 

126.  Perform  preventive  maintenance  on  automated 
coanuni cat Ions  systems  (using  MRCs). 

127.  Perfona  preventive  Maintenance  on  audio 
converters  (including  audio  digital  con¬ 
verters;  using  HRCs). 
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APPENDIX  S 


NAVY  RADIOMAN  JOB  DESCRIPTION  QUESTIONNAIRE  FOR  SUPERVISORS 


% 


NAVY  RADIOMAN 

JOB  DESCRIPTION  QUESTIONNAIRE 
(FORM  B:  To  be  filled  out  by  supervisors  of  firs^t-tera  Radiomen) 


Developed  by: 

Human  Resources  Research  Organization 
27857  Berwick  Drive 
Carmel,  CA  93923 

and 

Personnel  Decisions  Research  Institute 
43  Main  Street  Southeast,  Suite  405 
Exposition  Hall 
Minneapolis,  HN  55414 
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INTRODUCTION 


The  Navy  Is  conducting  a  project  to  find  the  best  ways  to  measure  the 
job  performance  of  first-ten*  Radiomen  In  the  Navy.  This  questionnaire 
is  an  early  step  In  that  project.  Its  purpose  is  to  learn  acre  about 
the  first-iene  Radioman  job  from  those  who  know  it  best- -first-term 
Radiomen  and  their  supervisors. 

Please  take  the  time  necessary  to  carefully  consider  each  ite®  and 
answer  it  to  the  best  of  your  knowledge,  fee  appreciate  your  help. 


PRIVACY  ACT  NOTICE 


Under  the  authority  of  S  USC  301  judigments  concerning  the  job  perfor¬ 
mance  of  first-term  Radiomen  are  requested  for  a  research  project  on 
performance  assessment.  - - -  - -  “***  —  — ~ 


.M 


m  JeiKH ^feilLAEmL^UO^EEEJM  MZBT-  It  will  be  used  by  the 
Navy  Personnel  Research  and  Development  Center  for  statistical  purposes 
only.  Return  of  the  questionnaire  constitutes  acknowledgme.il  of  these 
Privacy  Act  provisions. 


This  questionnaire  has  been  assigned  Report  Control  Symbol  OPHAV  5312- 
5 (OT > ,  which  expires  30  June  1585. 


f 
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SS&XAL  IMF8RK&TXGN 


Please  fif-lnt  your  answers  In  tht  spaces  provided  ©r,  where  applicable, 
place  i  check  mark  In  the  one  blank  that  best  applies  to  you. 


1.  Hint:  _ o _ _ _ _ _ 

2.  Social  Security  HuBber:  _ 

3.  Complete  Work  Hailing  Address: 


4.  What  Is  your  current  pay  grade? 


£-4 

E-5 

_  E-6 

£-7 

I _  £-8 

_  £-9 

_ __  Other  (please  specify:  _ __} 

5.  How  long  have  you  been  in  the  Navy? 

_  lts>  than  2  years 

_ _ 2-4  years 

_  4-8  years 

_ Here  than  8  years  (pleas*  specify:  _ } 

8.  For  how  long  have  you  supervised  first-ter*  R*dioa»n? 

_  Have  not  supervised  first- tens  Radloasn 

_  less  than  3  Booths 

_ _  3  oonths  to  6  »onths 

1_ _ 6  Booths  to  9  Booths 

_  9  Booths  to  1  year 

_  Hore  than  1  year 


So  on  to  the  next  page 
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7.  What  is  your  current  assignment? 


Ship 

Shore  installation 


enter  the  ship  type  and  hull 


number  (e.g.t  CVN68,  FF1083): 


check  the  type  below: 


_ _ General  Duty 

_ _ HAVCAMS  (also  check  one  of  the  following:) 

_  Fleet  Center 

_  Tech  Control  Facility  (including  SATCOM) 

Message  Center/Data  Base/Computer  Deck 

_  HAvCoMSTA  (also  check  one  of  the  following:) 

_ Fleet  Center 

___  Tech  Control  Facility  (Including  SATCOM) 

Message  Center/Data  Base/Computer  Deck 

_ Receiver  Site 

Transmitter  Site 
_ NTCC 

_ Other  (please  specify:_, _ ) 

8.  How  long  have  you  been  In  your  current  assignment? 


Less  than  3  months 
3  months  to  5  months 
6  months  to  9  months 
9  months  to  1  year 
More  than  1  year 


Go  on  to  the  next  page. 


INFORMATION  ABOUT  OOB  TASKS 


This  section  contains  a  list  of  first-term  Radioman  job  tasks.  The 
task  list  was  developed  by  examining  job  descriptions,  talking  with 
Radiomen  and  their  pervisors,  and  observing  first-term  Radiomen  on 
the  job.  This  list  covers  all  first-term  Radioman  jobs  at  all  instal¬ 
lations,  so  it  aay  contain  some  tasks  that  the  Radiomen  you  supervise 
do  not  perform. 


In  this  section,  you  will  answer  three  questions  about  these  tasks: 

--Is  each  task  performed  by  the  Radiomen  you  currently  supervise? 

For  each'  task  that  is  performed  by  them... 

--How  important  is  It  for  mission  success? 

--When  It  is  performed,  how  often  is  it  performed  incorrectly? 

As  you  fill  in  the  questionnaire,  consider  only  the  work  done  by  your 
subordinates  in  your  current  lob  assignment.  Do  not  consider  work  done 
by  those  you  supervised  in  previous  assignments. 

PERFORMED  BY  THOSE  CURRENTLY  SUPERVISED  Judgwent 

Read  the  first  task  and  decide  whether  it  is  performed  by  the  first- 
term  Radiomen  that  you  currently  supervise.  If  the  task  is  a  part  of 
at  least  one  Radioman's  job,  place  a  check  mark  next  to  the  task  In  the 
column  labeled  "PERFORMED  BY  THOSE  CURRENTLY  SUPERVISED".  If  the  task 
is  Mi  a  part  of  the  job  of  anyone  you  supervise,  go  on  to  the  next 
task  without  placing  anything  in  the  PERFORMED  BY  THOSE  CURRENTLY 
SUPERVISED  column. 


Go  through  the  entire  list  of  tasks  in  this  way.  DO  NOT  FILL  IN  ANY  OF 
THE  OTHER  COLU.MNS  UNTIL  YOU  HAVE  GONE  THROUGH  THE  ENTIRE  LIST  AND 
INDICATED  WHICH  TASKS  ARE  PERFORMED  BY  THOSE  YOU  CURRENTLY  SUPERVISE. 

Example 

Or  the  next  page  is  an  example  of  how  one  supervisor  Indicated  which  of 
three  sample  tasks  are  performed  by  the  first-term  Radiomen  he/she 
currently  supervises. 
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V 

if 

forma 
n  mx 
cmn: 

xnnmi 

ad 

1.  Encadc/fecod*  cell  signs. 

✓ 

■ 

> 

2.  Rig  smr^tncy  tnUtros. 

3.  Slot  aesstgts. 

mm 

- 

This  supervisor  went  through  all  the  tasks  and  indicated  which  ones  art 
performed  by  the  first- term  Radiomen  he/she  currently  supervises.  Be¬ 
cause  tasks  A  and  C  are  performed,  a  check  mark  was  placed  by  each  of 
those  tasks  in  the  PERFORMED  BY  THOSE  CURRENTLY  SUPERVISED  column. 

Task  B,  "Rig  emergency  antennas,*  is  not  part  of  the  job  of  any  first- 
term  Radioman  this  person  supervises,  so  he/she  did  net  place  a  check 
mark  in  the  PERFORMED  BY  THOSE  CURRENTLY  SUPERVISED  column  by  Task  B. 

This  supervisor  correctly  made  the  PERFORMED  BY  THOSE  CURRENTLY  SUPER¬ 
VISED  judgment  for  all  the  tasks  on  the  list  before  going  on  to  make 
any  other  ratings. 

IMPORTANCE  and  PERFORMANCE  ERRORS  Ratings 

% 

When  you  have  finished  indicating  which  tasks  are  done  by  the  first- 
term  Radiomen  you  currently  supervise,  go  back  to  the  beginning  of  the 
task  list  and  complete  the  IMPORTANCE  and  PERFORMANCE  ERRORS  ratings 
gnlx-for.. those  tasks.,  .that  you  Jo-b.e.ilR£QB?l£P  IX 

IHQSL^BEami  iUEEBmffl . 

The  IMPORTANCE  and  PERFORMANCE  ERRORS  ratings  are  to  be  made  by  filling 
in  the  gag  appropriate  rating  for  each.  A  different  rating  scale  is 
used  for  each  of  these  ratings. 

IMPORTANCE  Rating 

For  the  IMPORTANCE  rating,  Indicate  how  important  the  task  is  for 
mission  success.  To  do  this,  use  the  rating  scale  on  the  next  page  to 
rate  the  effect  that  incorrect  or  poor  performance  or.  the  task  would 
have  on  the  mission. 
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1  -  LITTLE  OR  HO  NEGATIVE  EFFECT 

2  -  MINOR  NEGATIVE  EFFECT 

3  -  SOME  NEGATIVE  EFFECT 

4  -  MAJOR  NEGATIVE  EFFECT 

5  -  MAXIMUM  NEGATIVE  EFfECT 

For  each  task  that  you  checked  as  PERFORMED  BY  THOSE  CURRENTLY  SUPER¬ 
VISED,  enter  the  am*  rating  (1  to  5)  that  best  describes  the  importance 
of  that  task  for  mission  success,  as  gauged  by  the  effect  that  Inade¬ 
quate  performance  on  the  task  would  have  on  the  mission. 

PERFORMANCE  ERRORS  Rating 

For  the  PERFORMANCE  ERRORS  rating,  Indicate  how  often  errors  are  made 
by  first-term  Radiomen  performing  the  task  without  assistance  or  direct 
supervision.  Use  the  rating  scale  on  the  next  page  to  make  the  PERFOR¬ 
MANCE  ERRORS  rating, 

1  -  SELDOM  (Performed  Incorrectly  2 OS  or  Less  Of  The  Tiiee) 

2  «  OCCASIONALLY  (Performed  Incorrectly  20-4QS  Of  The  Time) 

3  -  SOMEWHAT  OFTEN  (Performed  Incorrectly  40-60*  Of  The  Time) 

4  -  OFTEN  (Performed  Incorrectly  6Q-83X  Of  The  Time) 

5  «  VERY  OFTEN  (Performed  Incorrectly  80*  or  More  Of  The  Time) 

For  each  task  that  you  previously  checked  as  PERFORMED  BY  THOSE  CUR¬ 
RENTLY  SUPERVISED,  enter  the  £Q£  rating  (1  to  5)  that  best  describes 
how  often  errors  are  made  by  first -term  Radiomen  performing  the  task 
without  assistance  or  direct  supervision. 

Example 

After  checking  all  of  the  tasks  performed  by  his/her  first-term  Radio¬ 
man  subordinates,  this  supervisor  made  the  IMPORTANCE  and  PERFORMANCE 
ERRORS  ratings  aoly  for  those  tasks  that  had  been  checked.  The  ratings 
are  shown  on  the  next  page. 
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Task  A:  This  supervisor  believed- that  encoding/tiecoding  cell  signs 
incorrectly  would  have  a  «xi»uw  negative  effect  on  wission 
success.  Thus,  a  *5*  was  written  in  for  the  IMPORTANCE  rating. 
Errors  are  sade  aatthit  W-M*  tUd ifcJig£--by 

first- terw  Radioaen  when  they  perfona  this  task,  so  a  rating  of 
"3*  was  assigned  for  PERFORMANCE  ERRORS. 

Task  B:  This  task  was  not  checked  as  PERFORMED  BY  THOSE  CURRENTLY 
SUPERVISED.  Therefore,  the  supervisor  did  not  fill  in  the 
IMPORTANCE  and  PERFORMANCE  ERRORS  ratings. 

Task  C:  ‘Slotting  wessages*  incorrectly  was  believed  to  have  sctc-e  negative 
g£fec.t  on  wiss ion  success;  thus,  an  IMPORTANCE  rating  of  "3"  was 
assigned.  Errors  are  made  ficcislgnailr  (about  20-4Q*_gf  the  time! 
by  first-ten  Radiowen  perforsing  this  task,  so  a  *2*  was  written 
in  for  PERFORMANCE  ERRORS. 

Go  on  to  the  next  page  and  begin  the  questionnaire. 
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/  If  thfa  Utk  *«r*  parfamed  1ft- 
¥  carrtctly  #r  p*cr!y,  **tt  af'wt 

ff  M«U  tfctra  to  m  tka  afsslta? 

nifwci 

IT  TmVL  1  -  UTTLI  <*  IPO  ISMTm  EFFECT 

camsKTi?  t  -  km*  K«rm  effect 

«*«*!$»  3  •  SOME  K4AVm  EFFECT 
4  -  RU0*  KfiATm  EFFECT 
e  -  mxmm  m  effect 


fEifc-uswE  tax 

thii  task  It  h 
ha*  aflta  It  U  ptrfi 
facormtlj? 

1  •  Slum  (2M  nr  l 

2  -  PCCtf  IKULIT  (K 

3  •  SOBEVKAT  0F7EK  ( 

4  -  OfTtX  (tO-tOS) 

I  «  mi  OfTEH  (90S  i 


Sat  *7  LF  trtBtaf  tt»ri. 


Sat  *p  Hf  tranwi  tUnt. 


Sat  kp  YHF  traa*»1ttar». 


Sat  up  LF  ntcafvars. 


Sat  up  XF  racafvar*. 


Sat  wp  tHF  ractlvart. 


Sat  e?  UKF  trafijcairart. 


Sat  up  YHF  traatcafYart. 


Sat  «p  tatalllt*  traaicafaart. 


Sat  «p  tha  SSJH  tatolUU  rtcafrer. 


Sat  up  antaaM  c*vpler*/—Htccvpl«n. 


Sat  vp  crypto  t^ylpaant. 


Sat  up  talatypai. 


Chan$a  ptptr/rlhfcont  m  talatypat  t*i 
pristart. 


Patch  co— an  I  cat  I  gat  a^vlp— at  places 
tofcthar. 


JUfse/lovar  tstaasat. 


/Utaab1a/4itasst*kft  anta-os. 


Plat  ship  potltioflt  fsr  aatawu  mi  prim att 


Lot 4  prgfraa  Upas  fa  aipfiettc  tap*  reader. 


Lw4  pr»frM  tapes  fa  war  tapa  reader. 


Parfara  lata  ttartt  ae  crypto  a^elpaagt. 


Check  fra^vaficfct  far  artahlffty. 


(ha  chirp  s«w» der  t»  ^aUrafna  fr*$#e«cf*t. 


KfORIttCE 


ICTli 

2 


24.  This  1t«a  ft  for  keypunch  purposes  ©sly.  Do 
*ot  'jrri to  ratings  for  this  on*. 


21.  Conduct  radio  ehecks  {secar«/’.wrs*cvre). 


25c  Conduct  quality  control  tasta  e*  transait 
frequencies. 


27.  Road  journal  (or  channel)  leg  teletypes  to 
determine  systew  status. 


28.  Maintain  cceowni  cat  ions  status  board. 


29.  Pull  wessages  fro*  teletype  or  printer. 


X.  Strata  incasing  mssages  for  rtltvanca, 
precedence,  sacvrlty  classification,  ate. 


31.  Operate  reperforator. 


32.  Maintain  listening  watch  on  radiotelephone/ 
voice  circuits  (e.g.,  Harbor  Cemon,  HICOH) 
and  detentina  when  call  sign  is  received. 


33.  Receive  radiotelephone/voice  circuit  was¬ 
tages  and  transcribe  to  hard  copy. 


34.  Log  entries  in  the  Radiotelephone  Log. 

a 


35.  Recognize  and  properly  cowply  with  special 
Mssage  handling  procedures  ("personal  fora» 
Halted  dirtrifewtiofi,  high  precedence, 
etc*). 


35.  Monitor  channel  aywber  continuity  for  *es- 
sage  traffic. 


37.  Draft  broadcast  screen  requests  as  approp¬ 
riate. 


38.  Log  locating  aessages. 


3$.  Leg  aetseges  in  Scnarai  Message  Log. 


40.  Reproduce  aessages  on  photocopier  and  verify 
copy  quality. 


IWORTMCE  I 

tm<3wxa  E£ms 

J 

If  this  task  Mre  performed  In-  [ 

jfees  this  task  is  performed 

V 

correctly  er  poorly,  uhst  offectjfco*  eftas  i*  it  performed 

if 

rcRfcecs 

weld  there  be  on  the  mission? 

fftcerrectlyt 

IT  THOSE 

1  »  UTTtf  0*  m  HttTm  EFFECT 

1  •  SELPtXl  (205  *r  less) 

oaxEmr 

2  »  *!XC*  RECAT I YE  EFFECT 

2  »  CCCASISiALLT  {V5-4CXJ 

$y?£XY!SIS 

3  -  SOKE  RELATIVE  EFFECT 

3  -  $CK=UH*T  OTTER  (40-50*) 

4  *  MWCX  XESATIVE  EFFECT 

4  -  OFTES  ({$-§**) 

} 

5  «  MMIWH  RCCATIYE  EFFECT  | 

i 

%  *  VERT  PFTES  or  mere 

41.  Use  rooting  guide  to  determine  distribution 
or  routing  of  incoming  message?. 


42.  Use  Standard  Subject  Identification  Codas 
(SSIC)  Manual  to  correct  the  internal  dis¬ 
tribution  of  messages  oo  the  Video  Display 
Terminal  (VDT). 


43.  Manually  route  messages  to  appropriate  des¬ 
tinations  (e.g.,  slot  acssagcs,  post  on  road 
board,  etc.). 


44.  Verify  outgoing  rough  drafts  for  complete- 
mss,  accuracy,  format,  and  releasing  signa¬ 
ture. 


45.  Process  DO-1352  Fores  (data  pattern  message 
release  fores). 


45.  Leg  outgoing  messages. 


4?.  Enter  CCS  Fores  into  Optical  Character 
Reader  (OCR), 


4fi.  Use  Video  Display  Terminal  (VDT)  to  trouble¬ 
shoot  and  correct  OCX  Fores  that  Kara  been 
rejected. 


43.  Remove  OCR  Fores  from  Optical  Character 
Reeder  (OCR)  and  record  Processing  Signal 
Mumbers  (PSRs)  and  times  of  transmission. 


50.  Priori  tiro  outgoing  mestagas  according  to  j 
precedence  and  time  of  receipt. 


51,  Varlfy  smooth  messages  with  su^enrisor/ 
releasing  officers  prior  to  transmission. 


52.  Proofread  outgoing  messages  prior  to 
transmission. 


53.  Record  time  of  delivery  and  sorties  cross  on 

messages. 


1HFCRTAHCE 

VI f  this  task  vara  performed  In¬ 
correctly  or  poorly,  what  effaet 
If  would  thara  be  on  tha  mission? 
PERFORMED 

BY  THOSE  l  «  LITTLE  OR  KQ  RESAT I YE  EFFECT 
CURRENTLY  2  •  MIRQR  NEGATIVE  EFFECT 
SUPERVISED  3  -  SOME  RESATIYE  EFFECT 
4  -  MAOOR  RESATIYE  EFFECT 
S  •  MAXIMUM  RECAT  I  YE  EFFECT 


PERFORMANCE  ERRORS 

Whan  this  tatk  1*  performed, 
how  often  I*  it  parfenwd 
Incorrectly? 

1  -  SELDOM  (20X  ar  lass) 

2  -  OCCASIONALLY  (20-40*) 

3  -  SOKEVHAT  OFTER  (40-WX) 

4  -  OFTER  (CO-BOX) 

5  •  YERY  OFTER  (BOX  ar  more) 


54.  Process  eoanerclal  message  traffic  (perform 
word  counts,  determine  costs,  accapt  funds, 
etc.)  and  maintain  associated  flles/reports. 


55.  Close  out/connenca  all  radio  circuit  logs  at 
change  of  radio  day. 


56.  Prepare/transmit  ZIC/ZID  reports. 


57.  Maintain  communications  center  message 
files. 


58.  Maintain  general  message  files. 


5$.  Maintain  broadcast  files/logs. 


60.  Maintain  message  magnetic  tape  files. 


61.  Select/use  relevant  general  communications 
publications.  Instructions,  and  directions. 


62.  Make  corrections  to  and  page  check  communi¬ 
cations  publications. 


63.  Inventory  general  communications  publica¬ 
tions. 


64.  Type/foraut/edlt  messages  on  teletype. 


65.  Type/format/edlt  messages  on  Video  Display 
Terminal  (VDT). 


66.  Determine  appropriate  format  for  outgoing 

messages. 


67.  Type  messages  using  ACP-126  format. 


€3.  Type  messages  using  ACP-126  Modified  format. 


69.  Type  massages  using  JAMAP  128  format. 


70.  Type  messages  using  ACP-127  (NATO)  format. 


71,  Type  messages  using  Kara?  Correspondence 
(letter,  bmbo)  format. 


■ 


VIT  this  task  mu  performed  in¬ 
correctly  er  poorly,  what  effect 
If  would  there  bo  on  the  mission? 
PERFORMED 

BT  THOSE  1  »  LITTLE  0*  80  HE6ATIVE  EFFECT 
CUSXEXTL?  I  «>  Km*  RELATIVE  EFFECT 
SWEXT15SD  3  -  SOME  KESATfVE  EFFECT 

4  *  wu ox  ummi  effect 

S  »  MAXIMUM  KE6ATITE  EFFECT 


When  this  task  l*  performed 
how  often  It  It  performed 
incorrectly? 

1  -  SELDOM  (202  or  lost) 

2  •  OCCASiCailT  (20-40*) 

3  -  SOMEWHAT  OFTEN  {4£->i0*j 

4  -  OFTEN  (80-80*) 

5  ■»  TEXT  OFTEX  (80*  or  nn 


Type  and  format  messages  for  Optical  Char¬ 
acter  Reader  (OCR)  using  typewriter. 


Load  paper  tape  Messages  into  Video  Display 
Tenainal  (VOT)  for  editing. 


Select  and  change  to  appropriate  node  on  VOT 
(query,  router,  service,  etc.). 


Initialize  CUDIX  links  using  coeeund  tele¬ 
type  or  terminal. 


Establish/Maintain  communications  on  ship- 
shore,  ship-ship,  or  termination  circuits. 


Initiate/respond  to  challenge  and  reply 
procedure  on  radiotelephone  circuits. 


Initiate  transmission  authentication  pro¬ 
cedures. 


Establish/waintain  communications  on  task 
group  ORESTES. 


Enter  commands  on  VDT  to  establish  VFCT  cir¬ 
cuit  configurations  and  channelizations. 


Coordinate  communications  using  pony  loops 
or  order  wires. 


Operate  air  preheater  controls. 


Transmit  messagas  via  VDT  terminal. 


Transmit  paper  tape  messages  using  high¬ 
speed  tape  reader  and  command  teletype 
(CUDiX  system). 


Transmit  Messages  via  radioteletypc. 


Transmit  messagas  via  radietelephone/voice 
circuit. 


Encode/decode  radio  frequencies,  short  sen¬ 
tences,  and  voice  call  signs  using  approp¬ 
riate  publications. 
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IHPQRTMCE 


ruroftHttcs  gMOtt 


If  this  Utk  Mr*  performed  i*» 
correctly  »r  peeriy*  etoat  effect 
if  K5tt!4  there  be  m  tit*  mission? 
Ftftf 0BME9 

if  those  i  -  i mu  on  ao  hemtite  effect 

CWtRESTl?  I  -  HfSOR  KMTItt  EFFECT 

swtwiKft  3  -  Mm  mwim  effect 

4  -  HWOfi  HE  SAT  l  ft  EFFECT 
8  •  WttIMi  JK8ATIYE  EFFECT 


»xn  tM*  t*»k  Is  p§rf«»cad, 
hew  eftax'  is  H  performed 
fecerosily? 

i  •  sam  (esx  97  Us*? 

I  «  eCCASJBWAV  c»*4es) 

3  «  SOMEWHAT  Of  FEN  (4C-6S*) 

4  «  MTE*  (W-*QX) 

6  *>  TEXT  OFTER  (OCX  »»  Mr*) 


Receive  confidential  materials  (excluding 
CMS). 


Inventory  confidential  materials  (excluding 
CMS). 


Rout*  confidential  materials  (excluding  CMS) 
to  appropriate  personnel. 


Stor*  confidential  materials  (excluding 
CMS). 


Destroy  confidential  materials  (excluding 
CMS). 


Receive  secret  materials  (excluding  CMS). 


Inventory  secret  materials  (excluding  CMS). 


Store  secret  Mtcrlals  (excluding  CMS). 


Rout*  secret  materials  (excluding  CMS)  to 
appropriat*  personnel. 


Destroy  secret  Materials  (excluding  CMS). 


Receive  top  secret  materials  (excluding 
CMS). 


Witness  destruction  cf  top  secret  materials 
(excluding  CMS). 


Witness  the  receipt  of  CMS  materials. 


Witness  the  Inventory  of  CHS  materials. 


Witness  the  control  of  CMS  materials. 


Witness  the  storing  of  CMS  materials. 


Stor*  CMS  materials. 


Witness  th«  destruction  of  CHS  materials. 


Check  message  recipients'  Identification  and 
security  clearance  to  ensure  proper  classi¬ 
fication  for  level  of  traffic  being  picked  up 


If 

PERFOXHED 
If  THOSE 
CUXNENTIT 


If  this  tut  Mn  performed  In* 
correctly  or  poorly,  whet  effect 
weeld  there  be  ok  the  mission? 


IT  THOSE  l  •  LITTLE  OX  NO  NEGATIVE  EFFECT 
CUXNENTLT  2  •  HINON  NEGATIVE  EFFECT 
SUFERVISEO  3  ■  SOME  NEGATIVE  EFFECT 
4  •  HUON  NEGATIVE  EFFECT 
I  -  HMJWK  NEGATIVE  EFFECT 


When  this  t*ik  Is  porfennn 
how  eften  Is  It  performed 
ineorractly? 

1  »  SELWKI  (201  or  loss) 

2  •  OCCASIONALLY  (KMCS) 

3  *  S0KEK4AT  OFTEN  (40-WX 

4  «  OFTEN  (60-WX) 

f  *  VENT  OFTEN  (tOX  er  w»r 


This  Item  Is  for  keypunch  purposes  only.  Po 
not  writs  ratings  for  this  on#. 

Perform  preventive  maintenance  or.  receivers 
(using  feints nine*  Nocord  Csrds,  or  HRCs). 

Perform  preventive  maintenance  on  trans¬ 
ceivers  (using  HRCs). 

Perform  preventive  maintenance  on  transmU- 
ters  (using  HRCs). 

Perform  preventive  maintenance  on  internes 
(using  HRCs). 

Perform  preventive  maintenance  en  antenna 
couplers/aultlcouplers  (usir.j  HRCs). 

Perform  preventive  maintenance  on  teletypes 
(using  HRCs}. 

Perform  preventive  Mlntenance  on  switch¬ 
boards  (using  HRCs). 

Perform  preventive  maintenance  on  patch 
panels  (using  HRCs). 

Perform  preventive  maintenance  on  reperfor¬ 
ators  (using  HRCs). 

Perform  preventive  maintenance  on  paper 
shredders  (using  HRCs}. 

Perform  preventive  maintenance  on  copying 
machines  (using  IttCs). 

Perform  preventive  maintenance  on  Optical 
Character  Reeders  (or  OCXs,  using  HRCs}. 

Perform  preventive  maintenance  on  remote 
radio  telephone  operating  positions  (using 
HRCs). 


Perform  preventive  maintenance  on  emergency/ 
portable  radio  equipment  (using  IGtCs). 
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r—s  *- 


V 

If 

mfi mcs 

If  THOSE 
CWKttTLT 
PJHttl  StS 

JS9*I*E£ 

if  this  teat  wn  perferwed  in¬ 
correctly  or  peer!/,  what  effect 
wewld  there  he  m  the  aitasteal 

1  *  unu  m  m  auArm  tmci 

2  •  WHO*  XftftTm  EFFECT 

3  •  sac  wsAim  effect 

4  •  SM*  MfATIVt  EFFECT 

5  •  »S!3W  K£A?m  EFFECT 

am. 

*h*s  this  tea*  Is  perferaed, 
hw  eftea  Is  U  perfenead 
lacerrtctl/f 

1  -  3EUW.  (ZCS  er  less) 
t  •  0CCMICT5AU.T  (K-40S) 

3  -  MHEflAT  QF1IX  (41-MS) 

4  «  ottr*  («4-M3) 

5  -  ?TST  efl«  {«S  »r  sasrs) 

122.  Perfor*  prtvjntirt  Maintenance  on  automated 
Mtiigt  reproduetion/dlstrlbutlon  s/stsws 
(using  F*tC»). 

' 

123.  Clean  and  impact  tap#  heads. 

• 

124.  Perfor*  preventive  Maintenance  os  telegraph 
telephone  signal  converters  (using  WCs}, 

125.  Perfena  preventive  Maintenance  on  radio  sets 
(using  fffiCs). 

125.  Perform  preventive  Maintenance  on  autoweted 
coenunlcatlons  * /stews  (using  MRCs). 

127.  Perfona  preventive  Maintenance  on  audio 
converters  (Including  audio  digital  con¬ 
verters  i  using  mcs). 

• 

• 
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CRITICAL  TASK  SELECTION  STEPS  FOR  CATEGORY  I 


1.  Select  the  eight  tasks  from  the  category  with  the  highest  overall  mean  criticality 
values. 

2.  Replace1  these  tasks  if  they  fall  below  the  overall  mean  frequency  value  for  the 
category. 

3.  Replace  tasks  if  the  percentage  performing  (incumbents)  or  the  percentage  supervis¬ 
ing  (supervisors)  is  iess  than  50  percent. 

4.  Replace  tasks  if  the  frequency  mean  or  the  importance  mean  in  any  of  the  sampling 
ceils  falls  below  -1  standard  deviation  for  that  mean  for  the  tasks  in  the  category. 

5.  Replace  tasks  if  the  percentage  performing  or  percentage  supervising  falls  below  40 
in  any  sampling  cell. 

6.  Replace  tasks  if  the  mean  for  performance  errors  falls  below  the  mean  for  the 
category  or  if  the  mean  for  complicated  to  perform  falls  below  -1  standard  deviation 
for  the  category  means. 


1  Replacements  were  always  drawn  in  order  from  the  list  of  tasks  ranked  by  overall 
criticality  mean,  and  were  drawn  where  possible  to  meet  all  decision  criteria  prior  to  the 
current  one.  For  example,  in  Step  3,  the  next  unseiected  task  from  the  rank  ordering  on 
the  basis  of  criticality  mean  would  be  selected  if  its  frequency  mean  did  net  fall  beiow 
the  overall  frequency  mean  for  the  category  (Step  2). 


APPENDIX  D 

FINAL  SELECTED  CRITICAL  TASKS 


FINAL  SELECTED  CRITICAL  TASKS 


1.  Set  up  crypto  equipment. 

2.  Set  up  teletypes. 

3.  Change  paper/ribbons  on  teletypes  and  printers. 

4.  Patch  communications  equipment  pieces  together. 

5.  Type/for  mat/edit  messages  on  teletype. 

6.  Seiect/use  relevant  general  communications  publications,  instructions,  and 
directions. 

7.  Inventory  confidential  materials  (excluding  CMS}. 

8.  Inventory  secret  materials  (excluding  CMS.) 

9.  Destroy  secret  materials  (excluding  CMS). 

10.  Screen  incoming  messages  for  correct  address,  precedence,  security  classification, 
etc. 

11.  Recognize  and  properly  comply  with  special  message  handling  procedures  ("personal 
for,"  limited  distribution,  high  precedence,  etc.). 

12.  Monitor  channel  number  continuity  foe  message  traffic. 

13.  Use  routing  guide  to  determine  distribution  or  routing  of  incoming  messages. 

14.  Manually  route  messages  tc  appropriate  destinations  (e.g.,  slot  messages,  post  on 
read  board,  etc.}. 

15.  Verify  outgoing  messages  on  DD-173  for  completeness,  accuracy,  fofmst,  and 
releasing  signature. 

16.  Prioritize  outgoing  messages  according  to  precedence  and  time  of  receipt. 

17.  Proofread  outgoing  messages  prior  to  transmission. 

18.  Maintain  communications  center  message  files. 

19.  Route  classified  messages  (excluding  CMS)  to  appropriate  personnel. 

20.  Receive  classified  message  traffic  (excluding  CMS). 

21.  Perform  preventive  maintenance  on  receivers  (using  MRCs). 

22.  Perform  preventive  maintenance  on  transmitters  (using  MRCs). 


